Archive for Computing

| January 31, 2014
No comments

Week 3 – The State of the Union

This week was mostly spent on further research into UI.  I thought I’d take this opportunity to present a screenshot of where the VISTAS UI is at today.  It is currently a “bare metal” design; functionality has been kept at a minimum while the rendering engine has been worked on.

Vistas Screenshot

The Vistas UI, as it stands today.

Upon recommendation from Judy, I took a look at another visualization program called VisIt.  VisIt has been under development for many years by the Department of Energy, and in that time has become very feature-rich.  As you can see from the following screenshot, it is anything BUT “bare metal”:

VisIt Screenshot

A screenshot of VisIt, another visualization program.

As the scope of VisIt is presently much larger than VISTAS, I highly doubt VISTAS will ever need anything close to that many UI elements.  Nonetheless, I thought VisIt had some interesting features.  Namely:

  • Separate windows for loading/project management & visualization
  • Axis locking and zoom controls
  • Ability to change colors used dynamically via color table
  • Camera views – ability to lock, save & load
  • Save/Print Window for easy export of imagery

I also read an interesting paper: Current practice in measuring usability: Challenges to usability studies and research, Kasper Hornbaek, International Journal of Human-Computer Studies, Int. J. Human-Computer Studies 64 (2006) 79–102.  In it, Hornbaek acknowledges and evaluates the efficiency of various metrics for usability.  Particularly, he identifies the following measures of usability which can prove useful to my work on VISTAS:

  • Binary Task Completion — did the user finish their task?
  • Accuracy — how many errors did the user make while working towards their task?
  • Recall — how much information can the user remember after using the program?
  • Completeness — how complete of a solution did the user achieve?
  • Quality of Outcome — the user’s subjective satisfaction with the end result of program use

The differences between Accuracy, Completeness, and Quality of Outcome are subtle but very important.

In other project news, we are having a VISTAS teem meeting in Corvallis next week.  I am excited to visit with everyone, and am looking forward to the opportunity to talk to Margaret Burnett, an OSU professor who is an expert on UI and does some really cool research into machine learning techniques applied to UI.

Category: Computing, Senior Project - VISTAS Data Visualization | Tags: , ,
| January 22, 2014
No comments

Week 2 – The Spy++ Who Loved Me

This week, I learned about a fantastic tool integrated into Visual Studio called Spy++.  Many people are (painfully?) familiar with this tool, but it is new to me.  Spy++ allows you to target a UI component in a GUI application, and it will then show you a stack trace of that component.  It has already helped me a ton in figuring out how the VISTAS UI code works, as I can identify specifically what class each UI component represents and jump right to the code which generates it.  

Spy++ UI Trace

Just a taste of Spy++.

I also sat down with the application and had another brainstorm session on desired features that are within the scope of this project, which I’ll enumerate here.

VISTAS UI Ideas

Short term:

  • Fix the “Add” buttons on the Data & Visualizations tabs. Right-clicking is not intuitive to a new user. This could be simple as a small “+” or “Add” button next to
  • Project Visualizations/Project Data.
  • Add ability to delete items from Data & Visualizations tabs. Currently, removing a scene requires rebooting VISTAS.
  • Cleanup of VISTAS .ico file — remove white pixels from edge of image to beautify
  • Change default icon in titlebar of main window frame to be the VISTAS icon
  • Tooltips/mouse hover context information need to be added across the board.

Medium term:

  • Add “Help” section with integrated quick start guide
  • Revamp the Tool Bar. Re-evaluate overall layout, icon choices and functionalities present.

Longer Term:

  • Revamp the Menu Bar. Right now there is a bare minimum of functionality here, and several existing buttons don’t work.
  • “About” as its own item, and not nested under File — to be discussed.
  • Functionality to Save & Open projects needs to be implemented.  Nik has assigned himself to this.
  • Functionality to close the current project/create a new project needs to be implemented.
  • Adding/removing Viewers could perhaps be done in a more intuitive way.
  • Bundle a small sample data set with VISTAS so that a new user can experiment with the program without committing to downloading a large data set & going through all the steps of setting up the visualization. Ideally this would come in the form of a sample project include with VISTAS, so the ability to Save & Open projects needs to be implemented first.
  • Ability to take a screenshot of the viewer window, either via right-click or perhaps a button in the toolbar. Ability to copy screengrabs to clipboard for easy insertion into documents/emails/whatever.
  • Optional camera controls on the viewer window might be nice. At least a zoom control– Shift+mouse zooming is not always intuitive nor precise for fine camera movements.
  • Ability to click on colors in Legend and change them.

— whew!  I better get to work.

Category: Computing, Senior Project - VISTAS Data Visualization | Tags: , , ,
| January 15, 2014
No comments

Week 1 – Getting Set Up

I spent this week cloning the latest VISTAS build off of the bitbucket repo and making sure that my Visual Studio can successfully build it.  This was a bit tricky, as large open source software projects inevitably gather a lot of dependencies, but I believe that I have my workflow in place.

I also set up this blog, submitted my project proposal (see previous post) and did some online resource hunting.  I’ll list these resources here for me to come back to…

VISTAS Related:

VISTAS Project Blog
VISTAS Source Repository

wxWidgets Related:

wxWidgets Alphabetical Class Reference
wxWidgets Guides & Tutorials
wxFormBuilder – Open Source wx Designer

Research Papers of Interest:

User Interface Design with Visualization Techniques – A pretty basic paper, but introduces an interesting scoring rubric for UI analysis
Current practice in measuring usability: Challenges to usability studies and research – More interesting thoughts on UI quality heuristics

Category: Computing, Senior Project - VISTAS Data Visualization | Tags: , ,
| January 12, 2014
No comments

Project Proposal – User Interface Developer Immersion

Summary & Learning Objectives

For my senior project, I will be working collaboratively on a small development team working on VISTAS, a 3-D scientific data visualization tool.  This project is taking place under the mentorship of Judy Cushing and is funded by the National Science Foundation.  VISTAS has already been under development for several years, so a big challenge of my project will be learning my way around an existing large C++ codebase.  The rendering engine in the application is mostly complete; however, the tool is currently slightly buggy and can be somewhat unintuitive to use.

My role will be to focus on learning user interface design & programming with the goal of enhancing the usability of the program.  I also hope to learn about modern source control methods as well as obtain practice at working on collaborative software projects.  Throughout the course of my project, I will also be fulfilling additional developer responsibilities within the VISTAS team, including attending weekly online meetings and contributing to code reviews, bug fixes, and collaborative readings of select research papers.

Resources

The existing VISTAS UI uses a cross-platform library called “wxWidgets” for all of its graphical UI elements.  There are official wxWidgets tutorials as well as a reference page online– these will be my primary resources for this project.

VISTAS-related resources can all be found at the project’s webpage.

Implementation Plan

My plan initially is to target making addition and removal of visualization scenes more intuitive.  I also hope to add context-driven mouseover tooltips for user interface elements and a basic in-application help menu complete with a quick-start guide.

When it comes to exactly what I mean by “making addition and removal of scenes and visualizations more intuitive”, I had in mind:

  • Add ability to delete items from Data & Visualizations tabs.  Currently, removing a scene requires rebooting VISTAS.
  • Add “Add” buttons to the Data & Visualizations tabs.  Right-clicking is not intuitive to a new user.  This could be simple as a small “+” or “Add” button next to Project Visualizations/Project Data.

Beyond this, I’ve compiled a list of some user quality-of-life features that I think would be good additions.  The big two in my mind are the ability to Save/Open/Close projects and the ability to add annotations/pins to visualizations, but I’d like to work up to these.

Evaluation of Work

All of my code commits to the project can be viewed at my bitbucket profile (https://bitbucket.org/jmangue/profile/activity).  Furthermore, I will be keeping weekly online journals detailing my work on VISTAS.

Category: Computing, Senior Project - VISTAS Data Visualization | Tags: , ,
| September 6, 2013
No comments

My Internship at Wizards of the Coast

Wizards of the Coast

During the summer of 2013, I was the Performance Engineering intern at Wizards of the Coast, the West coast branch of Hasbro and maker of Magic: the Gathering and Dungeons & Dragons.  I performed in this internship under the mentorship of Ernie Nelson (Senior SDET, Customer Engagement Systems) and Sheryl Shulman (Computer Science Professor at The Evergreen State College).

According to the job description:

“Performance Engineering is an internship designed to expose the student to real-world experience as a Software Development Engineer in Test (SDET).  The student will work alongside an experienced SDET at Wizards of the Coast (a subsidiary of Hasbro) to build new performance test automation, maintain existing automation, execute performance tests and analyze test results.”

This blog chronicles my experiences.  See the following links to check out what I did in any particular week:

I learned a ton about all aspects of software testing, focusing on engineering automation and performance tests within a .NET-heavy environment.  Just seeing how real-world software teams work was invaluable, and will positively shape the start of my career moving forward.

During the course of my internship, in addition to software & performance testing, I also received experience & training in:

  • Using Visual Studio
  • Basic C# programming
  • Writing & executing simple SQL queries for data retrieval
  • Agile/Scrum methodologies
  • Source control (Perforce)
  • Installing new builds
  • Service Oriented Architecture principles
  • Software troubleshooting

I supported & reinforced my learning using the following texts:

  • Scrum: a Breathtakingly Brief and Agile Introduction (Johnson & Sims)
  • How Google Tests Software (Whittaker, Arbon and Carollo)
  • Professional Application Lifecycle Management with Visual Studio 2010 (Gousset)
  • Service Oriented Architecture for Dummies (Hurwitz)
  • .NET Performance Testing and Optimization: The Complete Guide (Glavich & Farrell)
  • Practical Load Balancing: Ride the Performance Tiger (Membrey, Hows & Plugge)
  • Advanced Web Metrics with Google Analytics (Clifton)

Overall, it was a fantastic summer and I am grateful to Ernie Nelson, Sherri Shulman, and Wizards of the Coast for the opportunity.

Category: Computing, My Internship at Wizards of the Coast | Tags: , ,
| September 6, 2013
No comments

Week 12 – Fond Farewells… and Cake!

This week marked my last week as an intern at Wizards of the Coast.

Knowing the end was near, I got a bit manic about wrapping up my work and ended up staying late almost every day this week.  I managed to tune up quite a bit of test automation and finally resolved a persistent issue with data file imports that had been plaguing me.

I released a new version of a C# tool which assists in debugging a particular database issue that was previously onerous to troubleshoot.  This tool has already proven quite popular within my department, and is my parting legacy.

I also wrote up a “QA Intern New Hire Resource Guide”, which was basically a knowledge dump of all the best links to helpful resources & tools I’d found, both inter-company and from the web at large.  Hopefully this will help out the next batch of interns to work at Wizards.

My team held a little farewell party for me:

Goodbye Cake

I count myself blessed that it didn’t say “Get Out Justin”.

It was a bit sad to pack up my office, but I am grateful that I was able to network with so many fantastic people in my time there, and I can check “Work at Wizards” off of my bucket list.

Category: Computing, My Internship at Wizards of the Coast | Tags: , ,
| August 31, 2013
No comments

Week 11 – Back to Baselines

My internship at Wizards is winding down, with just one more week to go. Hard to believe the summer has flown by so fast!

This week, I worked with Hazera (fellow performance engineer) to create some new baseline analytics for our load testing. The baseline is basically a load scenario which represents a typical day of virtual traffic for all of our web applications, which gives us a constant to run comparison runs against. I found myself grateful that I’d just taken a Statistics class, as that is what a lot of this boils down to.

I also did some troubleshooting for a web test plugin we use in our automation which extracts strings from our test email inbox. It wasn’t cleaning up after itself property, causing the inbox to periodically fill up. I ended up doing a little IMAP programming in C#, and I ended up additionally creating a load test plugin to clear the mailbox as needed. I was surprised at how easy the built-in C# classes make this.

One of the applications I’ve been helping do the performance testing on is called Planeswalker Points, and they had a major new feature release launch this week which went very smoothly. I couldn’t be prouder of the team!

This week marked my supervisor’s birthday. He is a big fan of Batman, so we literally covered his cube in Batman decorations and brought in a heap of desserts.

Next week, I’ll be going through my exit paperwork & interview, and doing my best to dump all that I’ve learned into documentation to help the next batch of interns along.

Category: Computing, My Internship at Wizards of the Coast | Tags: , , ,
| August 29, 2013
No comments

Week 10 – Loads More Load Testing

Load testing and performance analysis began this week for a major new release, which occupied most of my time.  I have been concurrently reading “.NET Performance Testing and Optimization: The Complete Guide” (Glavich & Farrell), which has cemented my understanding of performance testing.  I feel pretty confident that I could now walk into a new company and implement a performance testing program for web applications and services, from the initial hardware stages to the software side.  Basically, the rough process would go something like this:

  • Work with the Operations department to requisition some hardware to use for a load controller and load agents.  This doesn’t have to be anything fancy; it could even be old extra computers the company has laying around to start.  Ideally, though, you want your load setup to be as close in specification to the production servers as possible.
  • Set up the load hardware in it’s own network subnet with the system under test, to ensure that normal network traffic doesn’t color the load.
  • Install load controller & agent software on the machines.  The load agents are basically slave machines that actually spin up the simulated user load (this could potentially be thousands of processes), while the load controller simply manages and stores the resulting performance data in a central location.
  • Work with the business to determine desired performance metrics for the System Under Test, specifically:
    •  Desired user load
    •  Acceptance criteria for Response Times and other metrics
    •  Typical usage scenarios
  • Create data-driven test automation which reflects the various product usage scenarios, with validation in place to ensure that the requirements of the business are being tested.
  • Compose the test automation into a load scenario for the desired user load.
  • Execute the load scenario using the controller & agents.
  • Perform analysis on the results and determine if the System Under Test has attained acceptable performance.
  • If desired, move on to perform endurance, stress, and failover testing.

Obviously each step is actually a lot more intricate and detailed than that, but that’s a basic outline.  Note that the above is all textbook academic process, not necessarily Wizards’ process.

One of the most surprising things I have learned about performance engineering in my time in here is just how interdisciplinary it can be.  There are elements of psychology, statistics, and even philosophy in trying to simulate how the user will use the system.  It also really requires a broad knowledge of computer networking, hardware, databases, queueing theory, software development practices– you name it, and a performance tester can probably benefit from knowing it.  It seems like a career in which there is always more to learn.  Futhermore, since a performance tester is constantly analyzing the entire system from a bird’s eye view, he gains an instinct and insight for the System Under Test that others in the business do not, which leads to the performance engineer being a very valuable asset beyond just the fulfillment of his immediate duties.

I also tore through “Scrum: A Breathtakingly Brief and Agile Introduction” (Johnson & Sims) to deepen my understanding of Scrum and Agile methodologies.  It’s pretty interesting stuff, and is definitely superior to the old waterfall method for most kinds of software… but I had to chuckle a bit when I read that Scrum meetings are sometimes called “ceremonies”.

I have been training a new-hire SDET on what I know of web performance tests.  There is nothing like teaching someone else to build confidence in your own knowledge of an area.

I also got some more SQL practice this week, as I had to write a query to generate an answer key for the manual testers who are testing a Magic the Gathering exam site.  This was more work than it sounded like, as it required a 5-way table join and some casting.  I’m now feeling pretty good about my abilities to do basic CRUD operations in SQL, but hope to get yet more practice in the database realm in my remaining time here.

On an unrelated note– can I just say that this is possibly my favorite email I have ever gotten?

Best. Email. Ever.

Best. Email. Ever.

It couldn’t have come at a better time!

Category: Computing, My Internship at Wizards of the Coast | Tags: , , ,
| August 29, 2013
No comments

Week 9 – Saving The Environment

Most of this week was spent installing a new build of a major Wizards web service to our performance testing environment in preparation to do some load testing analysis.  I also worked to ensure that it functioned correctly and integrated successfully.  This required some remote troubleshooting along the way as well as some database work.I also did some independent research into load balancing, and reviewed some Architectural Design Documents (ADD) and Technical Design Documents (TDD).  It was pretty interesting to see how these are written in the real world.  Being a software architect would be a fascinating job.Sadly, this week marked the end of my remaining fellow interns’ time here.  We celebrated their departure with cake. I will miss the daily card games against them in the lunch room.

I am now the lone intern standing, and until such a time as they move someone into the old interns’ spaces, I have a huge office…

My workstation at WotC.

My workstation at WotC.

I just have three more weeks to try to work my way up to a corner office with a window!

Category: Computing, My Internship at Wizards of the Coast | Tags: ,
| August 22, 2013
No comments

Week 8 – WCF Heavyweight Championship

This week, I began to feel like I have finally hit my stride and begun to transition from a knowledge sponge into a competent, contributing member of the team.  I spent most of this week creating test automation for a Windows Communication Foundation service.  It was a bit tricky, as the particular service in question does not have a GUI but rather is a pure network messaging component, but fortunately there are tools out there (*cough*WCF Test Client*cough*) that ease the pain of figuring out the message schema.  By the end of the week, I had created a suite of component-based, data-driven automation that invokes the various API methods of a WCF service.  This automation will be used in the future for both performance and functional testing.

I also spent some time updating existing project files to use a newer version of the .NET runtime.

Company culture-wise, the department I’m in had a team outing this week.  We all went to a nearby park, where the heads of the project teams had rented out a space.  There was a virtual cornucopia of food available– pizza, snacks, and various other treats abounded.  We spent all day at the park, playing games and doing a photo scavenger hunt.  There was a bit of a competitive element, as there was a big pool of prizes up for grabs… mostly though, the day consisted of a lot of lounging in the sun and visiting with my coworkers.  It is always nice to get to spend time with people outside the office, and I’m grateful that I just so happened to be interning when this event took place.  Wizards definitely looks out for the happiness of it’s worker bees.

Category: Computing, My Internship at Wizards of the Coast | Tags: , ,
« Older Entries   Recent Entries »