Server Load Balancing Simulation

I read an interesting (to me) post about scaling a web application using the Heroku stack, and how they ended up having capacity issues not from the servers, but from the load balancing router.

Heroku recently changed the router from selecting the next available server to just selecting a server from the pool randomly and passing it without blocking.  Each server has its own blocking queue, in this case.  The justification from Heroku was that for a non-blocking server (like Node.js) this wouldn't be an issue, and should improve utilization.

But, the folks at RapGenius use Ruby on Rails, which is a blocking server, meaning they can process just one request at a time.

So I was reading the article, and they were complaining that they now needed to increase the number of servers, in order to meet the demand on their website.  I was starting to get antsy when they seemed to be speculating, but then they showed results from a simulation they ran in R, and that made me really happy.  Especially since they were able to put together statistics for their request processing times and identify a distribution.

I can't help but think how easy this model would be to build, run, and analyze in any of the available discrete event simulation tools we use, rather than use R.  But, R is free to use and our favorite discrete event simulators are not so much.  But, I wonder how many of these startups would ever consider using something like Simio to model their website infrastructure, and ultimately help themselves scale up.

Here's the post.

Etch a Sketch Charting

Every metrics board on every shop floor needs one of these.  Don't like the latest metric?  Shake it until you forget...

Someone should hook one of these up to MT Connect.

Lifehacker Link

eVSM User Group

There is a user group for eVSM on LinkedIn where I and the other developers of the software are members.  We actively read the activity on the group, and it's a great place to reach us for help with mapping problems, as well as submitting feature requests.  We also form feature review groups, which help drive improvement in the software.

Even though it's on LinkedIn, it's an open group, meaning anyone can at least view the contents, without having to have a LinkedIn membership.

eVSM User Group on LinkedIn

As an aside, if you have any suggestions for eVSM you can also submit them by using the Contact Me link on the side of this page...

Player Piano and MT Connect

I recently read Player Piano, by Kurt Vonnegut. The book is about a society that has had almost all manufacturing automated away, and the skilled labor relegated to fixing potholes, essentially being paid to do nothing.

The beginning of the book goes over how things are made. So at one point bright young engineers devised a way to record the movements of master machinists and play them back over again, obviating the need for machinists. Aside from the issue of how you'd go about making a new product or process change without a machinist to record, the idea had to be intriguing in 1952 when the book came out.

The really interesting part, though, is Vonnegut's description of the status display in the office in a factory (which had about 3 people working, total). The display simply shows a brass plaque for each machine, with a status lamp that would flash red if there was a problem.

The point of the book has little to do with how the factories worked, instead dealing with the resentment between the few people who still had meaningful jobs, and those that didn't. It did, however, get me to think a bit about what the point of MT Connect is. If you think you need MT Connect in order to keep all your machines working all the time, then that is completely misguided.

I suggest instead you read The Goal and put some thought into which machines need to be fully utilized. Spoiler: it's only a few of the many.

Which leads me to an encouraging thought: you don't need to connect all your machines in order to get value from MT Connect. You don't need a massive investment in Adapter installation and IT infrastructure. You could instead find your bottleneck resource(s) and monitor those to start correlating downtime with its root causes. Then you can fix the root causes and increase throughput without worrying about the 97% (made up statistic) of machines where it doesn't matter if they're working 100% of the time.

So the question I have now is what would you use to monitor bottleneck machines with MT Connect? Would it be as simple as a lamp for each machine that matters? Really, it could just be a smartphone app that shows a list of machines and flashes red for problems. Ideally it would route alerts to the right people automatically and keep some statistics on performance over time.

If no one beats me to it, maybe I'll build the app. As an aside, it would probably be easy to build using Firebase, which is excellent.

In conclusion, read The Goal and Player Piano and you'll be better off than you were.

Now For Something Not Entirely Different

I've quit my job at the CT Center for Advanced Technology (CCAT), after over six years.  CCAT was the first real job I had out of college, and I learned an incredible amount of skills and worked on incredible projects with incredible people.

I couldn't help it.

I've been given the opportunity to join the folks at eVSM as a consulting developer, helping them add new features to their product, which, if you've read much of this blog, I'm a pretty big fan of.  We'll also get into some new areas which I can't discuss, but it's very exciting for me.

The reason I couldn't help but leave CCAT was, the eVSM gig allows me to be my own boss, and take on contract work with anyone I want to work with.  The upsho is that I'm already contracting with CCAT on some MT Connect work, and look forward to helping them on interesting projects in the future.  I'll post a link when this first contract is done.

At the moment I'm pretty tied up with CCAT and eVSM work, but over time as I get acclimated to working in this manner (read: when I want, where I want, and what I want) I hope to create some great software products that will help people in manufacturing (and healthcare, services, etc...) be great at their jobs.  I also hope some of the five people who read this blog might consider taking on my services at some point.

This probably means I'm not going to be doing much of anything with QUEST, though I expect to be doing something with simulation very soon (probably with Simio though).  I also plan on continuing my participation in the MT Connect standardization process, because I truly think it will be a game changer in the field of Industrial Engineering.

MT Connecting to Twitter

Google Apps Script is awesome.  I've been using Google Apps for years, but until recently (when adding a snooze function to gmail) I had never used or really noticed Google Apps Script.  I'm going to abbreviate it to GAS, which isn't great, but I didn't pick the name.

So, GAS is basically the VBA of Google Docs, in that it's designed to help you integrate many of their services together.  I think GAS is even better than VBA because, for one, you get to write JavaScript, which, outside the browser context, is a joy to use.  It's great that you can just add new methods/properties all willy nilly, which, for someone who's been stuck with VBA for going on 9 years, is pretty great.

Another reason to love GAS (sorry) is that your scripts live "in the cloud", and can be run at periodic intervals.  Combine that with Google Docs' spreadsheet module, and you can do some interesting data collection without having to run software on your own computer.

And now I'll start getting to the point.  Because MT Connect is built on the same technologies as the web, we can use many of GAS's built-in XML tools to pretty easily parse out an Agent's data stream.  Also, remember how I said JavaScript lets you add new properties to objects willy nilly?  Well, the XML library adds XML attributes and nodes to objects, rather than making you have to call routines to check for them and get their values (I'm looking at you, VBA).

So, to show how cool this is, I put together a GAS App in a Google Docs spreadsheet that runs a function every minute, to grab the current power status of our Yasda H40i milling machine, and store the latest value in a spreadsheet cell.  Any time that value changes, the script calls a function to send a tweet on the Yasda machine's Twitter account (@CCATYasda5) denoting the new power status.  Ridiculous?  Absolutely.

At some point I may dig deeper into this tutorial and see if I can set up a UI where you can simply enter a few items (agent URL, device UUID, and data item(s) to monitor) and have it authenticate into your own Twitter account.  But the fact of the matter is, Google App Script made it very easy for me to pull in XML data over the web, pull up the previous status information, and handle authentication into Twitter with very little effort on my part, all for free, running on the web 24x7, which is why Google Apps Script is awesome.

Winter SImulation Conference 2011

Had another good year at the Winter Simulaton Conference last year in Tempe, AZ. I didn't get to play golf (which is lucky for everyone else on the course), and there was more rain than I'd like, but the conference itself was probably the best I've been too (out of 6). It didn't hurt that they had the conference mere miles from an in-n-out burger. But I felt that manufacturing is starting to get a little more popular at the conference, which I like.

I also was blown away by Simio for the 4th year in a row, with their announcement of the Risk-based Planning & Scheduling tool they've released. I also took their practitioners training course, which showed me even more how powerful, and great their product is becoming.

Forio was there again and was very interesting as always, as they now have support for running discrete event simulations in "the cloud." Very cool technology, which I think has a lot of potential for helping make simulation more of an every-day tool, on top of the education arena they seem to be popular in.

For the first time, I chaired a session of presentations, and was very impressed with the papers presented on auto-generated simulation models. I also presented in the Sustainability track on manufacturing sustainability (I think...it's been a couple months, and I guess I should have posted this sooner). Anyway, it was probably nothing terribly new for anyone who reads this blog, but I feel it was probably the last I'll talk about my work on CMSD, at least for a while:

If anyone has any comments or questions I'm always happy to respond. Though, the "contact me" link on the top right seems to get to me much faster than the comments. At least until I find a way to automatically subscribe to all comments...