Load testing and performance analysis began this week for a major new release, which occupied most of my time.  I have been concurrently reading “.NET Performance Testing and Optimization: The Complete Guide” (Glavich & Farrell), which has cemented my understanding of performance testing.  I feel pretty confident that I could now walk into a new company and implement a performance testing program for web applications and services, from the initial hardware stages to the software side.  Basically, the rough process would go something like this:

• Work with the Operations department to requisition some hardware to use for a load controller and load agents.  This doesn’t have to be anything fancy; it could even be old extra computers the company has laying around to start.  Ideally, though, you want your load setup to be as close in specification to the production servers as possible.
• Set up the load hardware in it’s own network subnet with the system under test, to ensure that normal network traffic doesn’t color the load.
• Install load controller & agent software on the machines.  The load agents are basically slave machines that actually spin up the simulated user load (this could potentially be thousands of processes), while the load controller simply manages and stores the resulting performance data in a central location.
• Work with the business to determine desired performance metrics for the System Under Test, specifically:
  •  Desired user load
  •  Acceptance criteria for Response Times and other metrics
  •  Typical usage scenarios
• Create data-driven test automation which reflects the various product usage scenarios, with validation in place to ensure that the requirements of the business are being tested.
• Compose the test automation into a load scenario for the desired user load.
• Execute the load scenario using the controller & agents.
• Perform analysis on the results and determine if the System Under Test has attained acceptable performance.
• If desired, move on to perform endurance, stress, and failover testing.

Obviously each step is actually a lot more intricate and detailed than that, but that’s a basic outline.  Note that the above is all textbook academic process, not necessarily Wizards’ process.

One of the most surprising things I have learned about performance engineering in my time in here is just how interdisciplinary it can be.  There are elements of psychology, statistics, and even philosophy in trying to simulate how the user will use the system.  It also really requires a broad knowledge of computer networking, hardware, databases, queueing theory, software development practices– you name it, and a performance tester can probably benefit from knowing it.  It seems like a career in which there is always more to learn.  Futhermore, since a performance tester is constantly analyzing the entire system from a bird’s eye view, he gains an instinct and insight for the System Under Test that others in the business do not, which leads to the performance engineer being a very valuable asset beyond just the fulfillment of his immediate duties.

I also tore through “Scrum: A Breathtakingly Brief and Agile Introduction” (Johnson & Sims) to deepen my understanding of Scrum and Agile methodologies.  It’s pretty interesting stuff, and is definitely superior to the old waterfall method for most kinds of software… but I had to chuckle a bit when I read that Scrum meetings are sometimes called “ceremonies”.

I have been training a new-hire SDET on what I know of web performance tests.  There is nothing like teaching someone else to build confidence in your own knowledge of an area.

I also got some more SQL practice this week, as I had to write a query to generate an answer key for the manual testers who are testing a Magic the Gathering exam site.  This was more work than it sounded like, as it required a 5-way table join and some casting.  I’m now feeling pretty good about my abilities to do basic CRUD operations in SQL, but hope to get yet more practice in the database realm in my remaining time here.

On an unrelated note– can I just say that this is possibly my favorite email I have ever gotten?

Best. Email. Ever.

It couldn’t have come at a better time!