A CLI Project With JCEV Using IDPM

Acronyms galore!  After learning about various software development tools and practices throughout this semester, I finally had a chance to apply them to a real group project.  This project involved creating a command line interface (CLI) which can be used to perform specific queries on the WattDepot server collecting data from the Hale Aloha residence towers.  We did this project in groups of three and used Issue Driven Project Management (IDPM) to keep the work flow moving smoothly.  Here I shall describe the process my group used and the resulting hale-aloha-cli-jcev system.

What is IDPM
First off, what is this issue driven project management?  Issue driven project management works by holding meetings often (every couple of days) to divide the current tasks into small work units that would take a day or two to complete.  These work units are called issues and are distributed between the group members.  The main principle behind IDPM is to consistently make, work on, and complete these issues so that every member knows what to work on and what to do next.  This keeps the flow of work consistent and helps to keep the project moving ahead smoothly and efficiently.  We implemented this methodology using the Google Project Hosting website which conveniently has an "Issues" page where we could manage our tasks.  We made rather extensive use of this page and ended up making more than 40 issues throughout the course of the project which aided us in getting everything done in a timely fashion.

Our project's issues page near the end of the project.

The CLI
So how does the project itself work?  The system itself is comprised of three main components.  The first is the main control class which runs the command line interface and grabs the user's input.  The second is a processor class which takes the user's input and tries to invoke the command that the user wants to run.  The third and final part contains the various commands that the user can invoke.  The main idea behind this approach is that each component is self contained and can treat each of the other components as a "black-box" as long as the developer knows what will be passed to the component he is working on.  For example, someone implementing an new command does not really need to know how the control class or the processor works as long as he understands that he should implement the provided Command interface and knows what will be passed to his command as explained in the interface documentation.  This makes the system somewhat modular to make it easy for developers to add to it.  The current release version implements all of the expected functionality and includes a total of six working commands: help (return a list of commands), quit (exit the CLI), current-power (returns the current power usage of given tower/lounge in kW), daily-energy (returns energy used by the given tower/lounge on given date), energy-since (returns energy used by given tower/lounge from given date to now), and rank-towers (rank towers from least to most energy used on given interval of days).

However, there are a couple quirks.  One major quirk is the quit command which needs communicating with the control class to stop the application.  This is a special case which required the quit command to throw a special exception which would be passed up to the control class as the obvious solution of using System.exit is a bad practice.  This works, but it does not quite fit the intent behind the design as both the control and the processor classes had to be modified to make it work.  Another quirk is the fact that the table generated in the processor class has to be modified to add a new command.  Ideally, the user could just add a new class and the processor could dynamically find it and add it, but we did not have time to implement a reflection-based processor class and resorted to using a hard-coded table.  As a result, anyone adding a new command would need to change the table in the processor class by adding a new put call to the table generation function.

Working with JCEV
As with any group project, you need a group and I had the pleasure of working with Jordan Takayama and Eldon Visitacion in team JCEV.  While it took us a little bit to adjust to each other's coding styles, we easily created a functional system before the due date.  This gave us time to check over each other's code and fix minor errors and inconsistencies like using "kWh" versus "kilowatt-hours".  Everyone also made a JUnit test case for each class with good coverage to ensure that everything is working properly.  As a result, I believe that our system contains quality software that has been thoroughly tested.

Conclusion
Overall, I believe team JCEV effectively used IDPM to create a CLI system.  We made good use out of IDPM to distribute and track the work that needed to be done which allowed us to finish with enough time to perform extensive checks and fix the tiny, non-system breaking errors.  Consequently, I believe that hale-aloha-cli-jcev is a quality software system that has been well tested to ensure that it works.  Not only that, but I have also gained valuable experience by putting all of the software engineering practices to use in an actual group project.  I have learned a lot from this project and I am definitely looking forward to next time!

Energizing Exercises: WattDepot Code Katas

If you have been keeping up with this blog, it should come as no surprise that energy-related research and software engineering go hand-in-hand and here we shall have our first glimpse at how these two seemingly unrelated fields can assist each other.  To recap, one way that software engineering can assist energy research is through data collection and analysis and software packages like WattDepot can be used to do just that (and more!).  As with the previous systems, we shall learn the basics of WattDepot through a set of simple exercises or katas to get some hands on experience as we get our feet wet.

---

Kata 1: SourceListing

Implement a class called SourceListing, whose main() method accepts one argument (the URL of the WattDepot server) and which writes out the URL argument and a list of all sources defined on that server and their descriptions, sorted in alphabetical order by source name.  Use the System.out.format method to provide a nicely formatted list. 

As the first exercise, Kata 1 was relatively simple.  It was made even simpler as the example program essentially does this already.  While the code itself was quick and easy, I was not sure if the sources retrieved from the server were always being retrieved in alphabetical order.  It took a while, but I eventually convinced myself that it was sorted and this uncertainty made exercise take about 25 minutes.

Kata 2: SourceLatency

Implement a class called SourceLatency, whose main() method accepts one argument (the URL of the WattDepot server) and which writes out the URL argument and a list of all sources defined on that server and the number of seconds since data was received for that source, sorted in ascending order by this latency value.  If no data has every been received for that source, indicate that.  Use the System.out.format method to provide a nicely formatted list.

The second exercise was rather straight forward as well.  It took a little bit of data manipulation to calculate the latency, but this was also shown in the example program.  The bulk of the time spent here was spent trying to figure out how to get the values sorted by latency.  After thinking about it for a few minutes I decided to implement a LatencyData class which stores a source name and latency value.  LatencyData implements the Comparable interface and contains methods to make it ordered on the latency value.  As a result, all I had to do was make a bunch of LatencyData objects, throw them into an ArrayList, use Collections.sort to get the results in the right order, and print them.  All of this took about 40 minutes since I had to override several methods for the LatencyData class.  However, I should have made a general sorting class to be used by the later exercises that require data to be sorted on a value other than name.  I chose to implement separate classes for each type of data in the later exercises (i.e. EnergyData and PowerData) and a general class would have saved time and effort.

Kata 3: SourceHierarchy

Implement a class called SourceHierarchy, whose main() method accepts one argument (the URL of the WattDepot server) and which writes out the URL argument and a hierarchical list of all sources defined on that server.  The hierarchy represents the source and subsource relationship between sources.

Here is where things started to get a bit more interesting.  Getting the subsources is rather simple using the getSubSources method, using the resulting object's getHref method, and then getting the subsource's name by retrieving the sub-string after the final '/', but making sure that the indentations were right and that subsources were not printed as top-level sources was a little bit trickier.  To print the subsources, I decided to write a recursive method that prints the name of the top-level source, then calls itself on each subsource with a larger indentation.  This way, it is guaranteed to print the hierarchy properly no matter how many levels of subsources are present.  In addition, the method removes any of the found subsources from the original source list, preventing the issue of them being printed again as top level sources.  All in all, this took about 45 minutes to figure out and implement.

Kata 4: EnergyYesterday

Implement a class called EnergyYesterday, whose main() method accepts one argument (the URL of the WattDepot server) and which writes out the URL argument and a list of all sources defined on that server and the amount of energy in watt-hours consumed by that source during the previous day, sorted in ascending order by watt-hours of consumption.  If no energy has every been consumed by that source, indicate zero.  Use the System.out.format method to provide a nicely formatted list.

Kata 4 was probably the most time consuming of them all.  While it seemed simple at first, getting the time stamps for the beginning and end of yesterday proved to be a little trickier than I thought.  To calculate the day of yesterday, I decided to use the java.util.Calendar class (though in hindsight, it would have been easier just to use the built in Tstamp functions).  I then used this Calendar object to create a string which would be used to make a time stamp to be passed to the getEnergyConsumed function.  However, I neglected to notice that the month field of the Calendar class was actually one less than the traditional numbering system (i.e. January is 0) so my time stamps were invalid.  This caused me to get a lot of BadXmlExceptions as the range I specified was outside of the stored sensor data.  It took me multiple sessions to figure this problem out and this exercise took at least 4 hours to complete.  On the bright side though, I was able to apply the same time stamp generation methods the following katas which made them much easier.

Kata 5: HighestRecordedPowerYesterday

Implement a class called HighestRecordedPowerYesterday, whose main() method accepts one argument (the URL of the WattDepot server) and which writes out the URL argument and a list of all sources defined on that server and the highest recorded power associated with that source during the previous day, sorted in ascending order by watts.  Also indicate the time when that power value was observed. If no power data is associated with that source, indicate that.  Use the System.out.format method to provide a nicely formatted list.

While this one sounded more complex than Kata 4 at first, it was relatively simple since I had the time stamps figured out.  The two main issues here concerned virtual sources (aggregates of non-virtual subsources) and finding the maximum power recorded.  Dealing with virtual sources can be tricky since they do not contain data points themselves and the subsources may not be synchronized.  However, this can be handled by sampling the data at large intervals to account for the discrepancies in synchronization (15 minutes in my case) allowing virtual sources to create data points by aggregating the data of its subsources.  The second issue of finding the maximum power is then easily brute-forced by iterating through all of the resulting data points and simply storing the one with the largest value.  Consequently, this one only took about 35 minutes to complete.

Kata 6: MondayAverageEnergy

Implement a class called MondayAverageEnergy, whose main() method accepts one argument (the URL of the WattDepot server) and which writes out the URL argument and a list of all sources defined on that server and the average energy consumed by that source during the previous two Mondays, sorted in ascending order by watt-hours.  Use the System.out.format method to provide a nicely formatted list.

Surprisingly, the final kata was not too difficult.  After playing the the Calendar class in the previous exercises, it was fairly straightforward to find the previous two Mondays.  The only tricky part was getting the previous week's Monday if today is Sunday or Monday since there would not be a complete data set for that week's Monday.  Other than that, getting the two energy consumed readings and averaging them was very simple.  As a result, this exercise was completed relatively quickly with a time of 25 minutes.

---

Overall, using these exercises as an introduction to WattDepot has shown me just how useful such tools can be in collecting and analyzing energy data.  With WattDepot, programmers do not have to go out of their way to get the data they want so they can spend their time doing meaningful analysis of that data.  While I did encounter a rather frustrating non-WattDepot related problem as I worked through these katas, the WattDepot API proved to be simple and painless to use.  So if you are interested in doing some energy data collection or analysis, WattDepot seems to be the way to go!

Energy in Hawaii and Software Engineering

Introduction
Hawaii is a unique place with its own unique set of problems.  This is especially true when it comes to the issue of electrical energy in terms of both generation and consumption.  So why would this be covered in a software engineering log?  Of course, this is an important issue for the residents of Hawaii to consider, but it is also producing some new and interesting software development opportunities.  Thus, let's take a look at some of the unique issues of Hawaii's energy situation and some of the efforts that are being taken to alleviate them.

---

Electrical Energy vs. Power
Before we begin, what is electrical energy?  Well, obviously we know that electricity is the thing that comes out of our outlets, but how do we measure it?  It turns out that there are actually two ways to measure electricity: energy and power.  Energy is the thing that makes our appliances run and makes electricity useful.  This is usually measured in kilowatt hours.  But why are our electrical appliances rated according to wattage?  These are actually ratings of power which tells us the rate at which these appliances consume electrical energy.  Consequently, the energy used is a factor of both the power being drawn by the appliance and the time it is running (energy = power * time) and the power is the rate at which the energy is being converted (power = energy / time).  Hence, these two related concepts are much more different than they appear to be at first glance.  For more information and a different explanation, please consult this video.

---

Issues
Now that we understand energy, what are the issues concerning electrical energy in Hawaii?  Of course, this energy must be generated in some fashion by the electric company and these are typically generated by using non-renewable resources.  While mainland power plants can use these resources, they often opt to use cheaper local resources when they are available.  However, due to Hawaii's unique location and available local resources most of its power generation comes from expensive imported oil and coal making energy costs much higher than on the mainland.  In addition, the mainland states make use of a single massive power grid to spread the workload and increase its efficiency.  On the other hand, Hawaii's geography prevents the creation of a single statewide grid, resulting in the use multiple much smaller and less efficient power stations to supply each island.  Therefore, Hawaii has a rather unique position on the energy front in comparison to other mainland states.

---

Efforts
With all of these problems, there must be a solution right?  Well look no further as the Hawaii Clean Energy Initiative (HCEI) has laid down what needs to be done to alleviate the aforementioned issues.  The HCEI aims to decrease energy usage by 30% while increasing clean energy generation by 40%.  This would lead to an astounding 70% increase in clean energy and be a huge step in improving Hawaii's energy future.  To do this, residents must increase their energy efficiency by following less wasteful energy usage practices such as keeping your refrigerator in a cool location or replacing your incandescent light bulbs with compact fluorescent light bulbs or Light-Emitting Diodes.  In addition, more energy would be generated through clean and renewable means such as using solar power or using wind turbines to capture the energy of the ever present wind.  As a result, both more efficient energy usage and the incorporation of local renewable energy resources are needed to help solve Hawaii's energy problems.

---

Software Engineering Opportunities
So where does software engineering fit into all of this?  For one, researchers will need to collect and analyze a lot of data to quantify the effectiveness of their energy efficiency efforts and they will need software tools to perform these tasks.  Consequently, they will need software engineers with energy knowledge to design and create these tools.  Furthermore, incorporating renewable energy resources will require smarter power grid technology.  These new sources must be added and removed intelligently as just throwing more energy into the grid would be wasteful as the non-renewable generators would still be going at normal capacity.   Conversely, removing too many of those generators would cause the entire grid to become overloaded and fail.  As a result, software will be needed to track both power generation and consumption to help incorporate these new energy sources safely and effectively.

---

Conclusion
Overall, Hawaii is a unique place with its own unique set of energy problems.  To solve these issues, Hawaii must both attempt to improve its energy efficiency and work on tapping the many local renewable resources.  However, both of these tasks will require the aid of software tools, providing software engineers with many research opportunities.  Hopefully you have gained a better understanding of Hawaii's current energy situation and can see how this seemingly unrelated field can be beneficial to the local software engineers.  So save energy and keep green.  Perhaps a software engineering opportunity might manifest itself as a result!