We Need More Power Captain!
A few lessons learned for others to note
Posted 1 year ago by Tom Maiaroto.
So I've been battling some issues with Virality Score the past week here. I like to post about both the good times and the not so good. This is certainly a challenge, but for as stressed as it's making me - it's not the end of the world. It's a problem with a solution. Unfortunately, I need to put food on my table so it's a bit difficult at the same time.
A good word of caution to anyone thinking about that fancy new SSD based hosting company Digital Ocean. Yea, their servers are cheap. Yea, they are SSD based and have high I/O. However, their bandwidth is less than stellar.
What was happening was I couldn't gather the data for Virality Score fast enough. Digital Ocean's network just wasn't fast enough (or they limit your server). Rackspace would have been a little better, but only if I was using a higher end (and more expensive) server. Rackspace gives you more bandwidth the larger your virtual server is.
What did I do? Amazon. Their pipes are massive. Even a micro instance is outperforming a 4GB server on Digital Ocean. However, it will need to be upgraded because the resources (CPU and RAM) aren't quite enough. The problem? It's far more expensive than Digital Ocean and Rackspace.
I guess the saying, "you get what you pay for" is pretty much true. Only in the hosting game it's extremely deceptive and hard to tell. For one operation you could be given more than enough resources, but for a differnet one, far too little resources. So when you go around looking for web hosting, please be sure to try out various vendors. I can't stress this enough.
It was a comabination of server power and me stress testing the application. I'm watching a ton of sites. Far too many for my budget (but I wouldn't have known how much was too much any other way), but my goal is to build the software to be more efficient so that the operating costs are as low as possible.
That said, I had a rough data set back at just the wrong time. More people are looking at the site, I'm starting to get press coverage, etc. I'm still up til 2 and 3am every night working away all while meeting with people, pitching, and designing the product. Literally doing 3+ full-time jobs. Oh right, and then freelance to pay the bills on top of that.
So if you've noticed any strangeness on Virality Score, I just wanted to apologize and let you know what's up. Also know that I'm working on resolving the issues...But it may just come down to only showing a fraction of the data (or a shorter window) for free. So in order to see more, I'll need to start charging for the information. Or get funding or something. It will scale and it won't take a ton of servers, but unless it's a business - it's a very big cost to an individual. I'd love to provide information to people for free, but I'm starting to fear that the demand is simply too great.
The hard part here is that it's not an instant thing. Everything is bound to time. Make one change and it'll literally take hours to know if it was the right thing to change and there's maybe some small ways to test and double check (simulations of data - this goes beyond your normal test cases which I have!), but again not for the time I have on hand.
The good news is that I think I've started to clear up some of the issues and data should be flowing again. In a few hours to a few days, things should look good again.
Filed under social mediavirality score
Search for Posts
Popular Labelssocial media virality score general internet web development reviews and opinions php web tools hosting lithium web design
- 16 Jun Social Media Analytics on a Budget
- 15 May Thinking About Using Laravel?
- 22 Apr Why Dart?
- 16 Jan PHP 5 Framework Comparisons: Bootstrap
- 02 Jan GitHub Issues Feels
- 23 Dec News Ox: Building a Better News Aggregator
- 18 Dec Discovery and Ranking
- 12 Dec Viral News Search
- 06 Dec Flexible Twitter Bootstrap Grid
- 04 Dec Problems With Google Analytics