IMPORTANT TECHNICAL NOTICE

Posted August 21st, 2016 by jared

For those who haven't noticed, Mintees.com (it's application and associated database) has fallen victim to a series of incredible remarkable technical issues beginning Friday night around midnight.

Long story long:

One of the primary servers on our network (one that houses Mintees and its database) experienced drive failure. This is not normally a big deal, the drives are set up in a RAID configuration to allow us to replace drives when necessary.

Once the drive was replaced, the data center did a file system check and discovered that the system was riddled with unrecoverable errors and we needed to do a restore from backup.

Normally this would not be that big of a deal, because our data center and server administrators have redundant back ups up to 7 days and then large chunks by week and even by month. Unfortunately, they discovered that every back up since January 5 has been corrupt and they never did anything about it or let us know.

Long story short, we went back in time to January 5, 2016.

We take this very seriously, this is a really big deal to us. In fact, there are a handful of other apps that power AKT that suffered from this and we are not looking forward to going back and repopulating missing data.

For the avoidance of doubt, the following are examples of what people shall be experiencing:

- No accounts created from January 5 forward currently exist

- Account settings you've changed since January are gone

- Tees, comments, posts, jobs, likes, approvals, denials after January 5 are gone

- Users who have had their profiles deleted after January 5 are back.

You get the idea.

---

While it is possible we may be able to recover something, the chances are slim to none. We've spent the entire weekend tirelessly night and day just to get things operable again.

On a personal note, in my ten years of running various applications across any number of server configurations, I have never experience such a string of bad luck and problems in one scenario by far. It is not a situation we will soon forget and surely one that will influence the future server configuration of Mintees.

We will update this thread if anything changes.

Thanks for your patience and understanding. We know this is a brutal situation to put our users through, but at this point we can only work with what's in front of us.

Jared, Cory & the entire AKT team

65 Comments

Add Comment

jared said about 4 years ago

An additional thought...

We know how much faith you put in us (Mintees.com) to store your artwork, thoughts, resources and profiles. We really let you down and sincerely apologize for it.
atomicchild said about 4 years ago

sad times I just hope this isn't the final nail in the coffin as things had slowed way down.
fuggart said about 4 years ago

re-uploaded as many of mine as I could remember... not something I'd rush to do again as it took a good half hour this morning to re-mock and upload!
joemaggots said about 4 years ago

thanks , I guess my design was deleted
dbdesign said about 4 years ago

Shit happens, dudes, could have been worse ;)
Cameron Latham said about 4 years ago

The queue is up to two pages, but upon clicking the second page, a glitch happens where it's still all submissions from the first page.
jared said about 4 years ago

I see page 1 and 2 as normal!
Skull With Hair said about 4 years ago

i want my money back!
Mgraphic said about 4 years ago

already promote my artwork a few day ago before technical error...and now it's all gone hh lol
jared said about 4 years ago

Mgraphic said:already promote my artwork a few day ago before technical error...and now it's all gone hh lol

Go ahead and upload and re-promote it and I'll refund your money through paypal for this new transaction! giving you an extra two weeks (or whatever the timeline is)!
jared said about 4 years ago

Skull With Hair said:i want my money back!

I keep trying to paypal you $0.00 but it won't let me.
miles to go said about 4 years ago

I've been a big fan of this site and of course emptees, but unless its actively making you guys money, maybe it's time to let the site go. The engagement on here has gotten really low and the time to keep it going without doing some sort of huge overhaul seems like a lot. I say it with love, but maybe the site needs to get a viking funeral.
jared said about 4 years ago

miles to go said:I've been a big fan of this site and of course emptees, but unless its actively making you guys money, maybe it's time to let the site go. The engagement on here has gotten really low and the time to keep it going without doing some sort of huge overhaul seems like a lot. I say it with love, but maybe the site needs to get a viking funeral.

This site is not a money maker for us, never was. It pays for its hosting, sort of. It's more about keeping it around for the tens of thousands of visitors per month. The jobs section is really active and so many users love posting tees, I just can't justify taking it down because people don't comment as much.

This site affects the lives of more people than those who actively talk--at least that's what the numbers say, you know?
fuggart said about 4 years ago

jared said:
miles to go said:I've been a big fan of this site and of course emptees, but unless its actively making you guys money, maybe it's time to let the site go. The engagement on here has gotten really low and the time to keep it going without doing some sort of huge overhaul seems like a lot. I say it with love, but maybe the site needs to get a viking funeral.

This site is not a money maker for us, never was. It pays for its hosting, sort of. It's more about keeping it around for the tens of thousands of visitors per month. The jobs section is really active and so many users love posting tees, I just can't justify taking it down because people don't comment as much.

This site affects the lives of more people than those who actively talk--at least that's what the numbers say, you know?

I agree that even though the participation/conversation is in the dumps and no one at all <3s the designs any more it does still attract clients and business... I still upload knowing that it's being seen. that being said if it was costing me money personally I'd be pulling the plug in a heart beat
Cameron Latham said about 4 years ago

I believe I have Mintees to thank for most, if not all, of my longest standing clients. So, it'd be a huge bummer to see it go.

There's really no other place for people in our industry since BandJob called it quits. Dribbble is packed full of the people that take design way too seriously and is generally just kind of a circle jerk. Foalio was cool, but no one cared, so they pulled the plug. Behance... Well, I don't think anyone actually uses Behance?

Mintees is the last haven for music industry designers aside from a few Facebook groups scattered about, but they have the same circle jerk tendencies of Dribbble, so it's difficult to find any real value there.

But I do have to agree that if the site is only costing the owners money, then it'd be understandable if it were to be let go. Although I do now appreciate Mintees even more because of that fact.
atomicchild said about 4 years ago

Can I get auto approval and I will upload a bunch this weekend. The approval process is way slow and not many people are actually judging the shirts these days it seems.

I agree with everything Cameron said this is like a beacon for the Music t-shirt design world.
jared said about 4 years ago

atomicchild said:Can I get auto approval and I will upload a bunch this weekend. The approval process is way slow not and not many people are actually judging the shirts these days it seems.

I agree with everything Cameron said this is like a beacon for the Music t-shirt design world.

Done
atomicchild said about 4 years ago

Thanks man!!!
nonzip said about 4 years ago

please Can I get auto approval
it make me With power In Drawing A lot
Please, I must the chance

Thanks man!
dobi said about 4 years ago

nonzip said:please Can I get auto approval
it make me With power In Drawing A lot
Please, I must the chance

Thanks man!

i gave you auto approval, but you have to promise me you will must the chance.
nonzip said about 4 years ago

dobi said:
nonzip said:please Can I get auto approval
it make me With power In Drawing A lot
Please, I must the chance

Thanks man!

i gave you auto approval, but you have to promise me you will must the chance.

Thanks dobi
Thank you very much for the chance
I promise!!!!!!!
BodilPunk said about 4 years ago

jared said:
atomicchild said:Can I get auto approval and I will upload a bunch this weekend. The approval process is way slow not and not many people are actually judging the shirts these days it seems.

I agree with everything Cameron said this is like a beacon for the Music t-shirt design world.

Done

This is really, really bad news. 2 of my designs are 'tee of the month', and some of them are 'tee of the day'. Now all has gone. I think I deserve to get auto approval too.
bsteczdesigns said about 4 years ago

atomicchild said:Can I get auto approval and I will upload a bunch this weekend. The approval process is way slow and not many people are actually judging the shirts these days it seems.

I agree with everything Cameron said this is like a beacon for the Music t-shirt design world.

I've been voting every day as I can, but I agree the overall voting seems slow. I'm not sure if all the voters that were determined recently (I say recently but it may of been last year now?) have kept up with the site.
atomicchild said about 4 years ago

Sorry I started that I want auto approval train.......
Mgraphic said about 4 years ago

jared said:
Mgraphic said:already promote my artwork a few day ago before technical error...and now it's all gone hh lol

Go ahead and upload and re-promote it and I'll refund your money through paypal for this new transaction! giving you an extra two weeks (or whatever the timeline is)!

yeah thanks bro that was awesome.. will re-upload it again soon,, thank you so much \m/
jared said about 4 years ago

Who had voting privileges before that do not have it now? You would have been granted this sometime after January 5, 2016. And before you lie, Rob will be here to confirm and set them back :)
Cameron Latham said about 4 years ago

bsteczdesigns said:
atomicchild said:Can I get auto approval and I will upload a bunch this weekend. The approval process is way slow and not many people are actually judging the shirts these days it seems.

I agree with everything Cameron said this is like a beacon for the Music t-shirt design world.

I've been voting every day as I can, but I agree the overall voting seems slow. I'm not sure if all the voters that were determined recently (I say recently but it may of been last year now?) have kept up with the site.

Yeah, same. Good call though.

Voting was relatively efficient pre-crash. I think we need more voters rather than new people on auto-approve. Some rando just uploaded nearly 30 designs and they could use a thumbs down.
EricAbstruse said about 4 years ago

I've noticed a lot of designs get stuck at the bottom of queue for voting. I agree with Cam - more (dependable) voters rather than auto-approve. Is there a way to limit people from uploading their entire portfolio at once?
fuggart said about 4 years ago

I try to keep voting but most times I do I upvote one or two designs out of about 50 so it can feel like an uphill challenge some days. is there any way of streamlining the voting process? for example does it need to refresh the page every time i thumb up/down a design or can you just tick the vote for them all and then click "send" or something at the end?
Cameron Latham said about 4 years ago

fuggart said:I try to keep voting but most times I do I upvote one or two designs out of about 50 so it can feel like an uphill challenge some days. is there any way of streamlining the voting process? for example does it need to refresh the page every time i thumb up/down a design or can you just tick the vote for them all and then click "send" or something at the end?

Definitely into this idea.
bsteczdesigns said about 4 years ago

Cameron Latham said:
fuggart said:I try to keep voting but most times I do I upvote one or two designs out of about 50 so it can feel like an uphill challenge some days. is there any way of streamlining the voting process? for example does it need to refresh the page every time i thumb up/down a design or can you just tick the vote for them all and then click "send" or something at the end?

Definitely into this idea.

Agreed as well
jared said about 4 years ago

fuggart said:I try to keep voting but most times I do I upvote one or two designs out of about 50 so it can feel like an uphill challenge some days. is there any way of streamlining the voting process? for example does it need to refresh the page every time i thumb up/down a design or can you just tick the vote for them all and then click "send" or something at the end?

Bros...

Just hold Command + *click* when you vote something up or down. It opens the result in a new tab. (Control + *click* for windows weirdos).
jared said about 4 years ago

i've also lowered the threshold for approval/denial.
THEGOODSHIT said about 4 years ago

thanks man
Namsing said about 4 years ago

EricAbstruse said:I've noticed a lot of designs get stuck at the bottom of queue for voting. I agree with Cam - more (dependable) voters rather than auto-approve. Is there a way to limit people from uploading their entire portfolio at once?

Agree !
xod03 said about 4 years ago

Hey guys. If you need more voters, i'd be down to help out
13UG-13th said about 4 years ago

Another thing please fix the login system. I have to login every time when i visit the site and sometime when i click to see the design or type the comment the system always log me out :)
zamboaga said about 4 years ago

A donate button should be created. Surely many people are willing to give back to this community including me.
Thank you for everything Mintees!
MrSnake said about 4 years ago

Maybe I can help with some programming. I remember you said mintees was built with cakePHP.
fuggart said about 4 years ago

jared said:
fuggart said:I try to keep voting but most times I do I upvote one or two designs out of about 50 so it can feel like an uphill challenge some days. is there any way of streamlining the voting process? for example does it need to refresh the page every time i thumb up/down a design or can you just tick the vote for them all and then click "send" or something at the end?

Bros...

Just hold Command + *click* when you vote something up or down. It opens the result in a new tab. (Control + *click* for windows weirdos).

Word... that worked great! much easier to keep up with voting now. Just a thought... donghan should be on auto approve now surely? dude produces solid stuff every time

http://www.mintees.com/people/208202-donghan/