Subquery.net

The following is a step-by-step guide on how to set up a Data Input Method, Data Template, Graph Template, Data Source, and Graph for Cacti.  If you don’t know what Cacti is, check it out here.

Recently I’ve been having trouble with the number of threads that apache is spawning via prefork.  I wanted to keep a constant monitor on just how many threads apache needed to spawn in order to handle requests.  I also wanted to be able to see what times of day saw the most traffic and what the most and least number of active threads was at any one time.

Since I already had Cacti installed and running nicely on my server, the solution to this problem was obvious.  I set out to create a custom data source and graph for cacti which would poll for the number of active apache processes every 5 minutes.  To get this data, I use the following shell command:
ps -aux | grep -c www

This command returns the number of processes which are running under the www (apache) user.  The following are instructions on how I used this simple console command to produce an informative graph in cacti:

First, navigate to the “Data Input Methods” page and click the “Add” link to add a new data input method.  You can enter any name you’d like for the input method.  For our purposes, you’ll need to set the “Input Type” to “Script/Command”.  This setting allows you to execute any console command or run any script.  The “Input String” field is the path to our script, along with input parameters.  If you are using the results of a console command, simply type that command here.  If you are executing a script, you will need to specify the full path to the script.  Since we’re only executing a simple shell command, all you have to do is type in the following:
ps -aux | grep -c www

After clicking the “Save” button, you’ll be brought to the Input Method’s settings page.  Here you can define names for your return values as well as define input values (these will be covered in a later post).  For this simple example, we have no input fields and only one output field.  Go ahead and click the “Add” button in the “Output Fields” section.  This will bring up a new page with options for the output field name and friendly name.  The output field name is the one returned by your console command or script.  This value MUST EXACTLY match your command’s output.  However, this is OPTIONAL for commands which only return a single result, such as the shell command we are running.  If your script or shell command only returns a single value, you can type anything you want here.  Be wary of the character limit for this field and note that no special characters are allowed.  The friendly name can be anything you’d like it to be.  Click the “create” button now.

Click the “Save” button to return to the Data Input Methods listing page.

Now that we have a data input method set up, we can configure a Data Template.  As we did with the input method, navigate to the “Data Templates” page and click the “Add” link.  Here you can enter anything you’d like for the Data Template name and Data Source name.  Select the Data Input Method created in the previous step from the drop-down box.

The “Data Source Item” section is where you will enter all of your input method’s output fields.  For the first one, go ahead and enter the name of our only return value, “numProc”.  This value DOES NOT have to be the same as it was in the input method.  This can be a friendly name.  Leave the minimum value as-is and change the Maximum Value to 1000 (or whatever maximum value is appropriate for the range of values you expect this field to return).  Leave the “Data Source Type” as “GAUGE”.  This simply means that the values are collected as discrete points.  See the cacti documentation for more information on other types.  Now click the “Create” button.

Once the page reloads, you’ll see that our Data Source Item has a new option for “Output Field” and it has already been set to the only return value from our input method.  You can now click the “Save” button at the bottom.

Next we create a Graph Template.  As we did for “Data Templates”, navigate to the “Graph Templates” page and select “Add”.  Enter any appropriate name you desire for both the Template Name and Graph Title.  All other values on this page can be left default except for the “Upper Limit”.  You’ll want to change that to be the maximum value you expect to plot on your graph.  In our case, we’re going to set the value here to 1000.  Now click the “create” button.  The page will reload and you’ll be presented with yet more options for the graph template.  The first thing we’ll do here is add an item by clicking “Add” in the “Graph Template Items” section.

In the Graph Template Item creation page, you’ll first want to select the data source for your graph item.  If you’ve done everything correctly so far, you should see your Data Template and input field in the drop-down list.  Once your data source is selected, you can set the other display values to anything you’d like.  I set the color to red and selected “LINE1″ for the Graph Item Type.  Finally, the “Text Format” format field value is the label that will be displayed in the graph legend.  Type “Number of Processes” (or whatever describes your graph item) into this field.  The cacti documentation provides more detail on what all of these options do.
Click the “create” button.
In addition to graphing a nice line, I also like to see what the last graphed value was in numeric form.  To do this, click the “Add” button again to add another graph item.  Select the same Data Source as before, but this time select GPRINT as the item type.  Select “LAST” for the consolidation function and type in “Last” for “Text Format”.  If you would also like average, max, and min values, just repeat the previous steps; changing “Consolidation Function” to AVERAGE, MIN, or MAX as appropriate.

Once you’ve finished with your graph settings, click the “save” button at the bottom.

Now that we have all of our templates set up, we need to create a data source for our server.  Naviage to the “Data Sources” page and select “Add”.  There aren’t many options on this page.  Just select the data template we created earlier and then select the host machine and click “create” and then “save”.

Finally, all the pieces are in place and we can create an actual graph for our server.  Navigate to “Graph Management” and click “Add”.  Once again, select the graph template and host and click “create” and “save”.

Now the next time your cacti process updates its data sources, you should have a nice graph of apache processes!

Apache - Number of Processes

I’ve come across a few very useful articles and pages on the MSDN web site and elsewhere on the web which have helped me greatly in the development of my own form control.  These resources are outlined below:

How to: Create an Owner-Drawn List Box – Very useful if you are creating a form control from scratch.  This particular article is all about creating a list box control.  It helped me out a lot in the creation of my text box control, most notably the bits on using Vertical Scroll Bars.

Windows Forms Controls in the .NET Compact Framework – More or less everything you’ve ever wanted to know about windows forms controls and the .NET CF.

GDI Functions – All the reference you need for actually drawing your form controls (or anything else).

Using GDI+ on Windows Mobile – An article on using GDI+ functions in windows mobile. Not recommended if you are developing for older devices.

User-Drawn Controls – Additional information for creating user-drawn controls

Custom Controls in Visual C# .NET – While not aimed at mobile development, I still found this article very helpful in understanding how form controls work.

This post will cover how to use off-screen buffering for User Drawn Windows Forms controls.  This post applies to both Windows Mobile and standard Windows form controls.

I just ran across another one of those problems which should have an obvious solution, but can be frustrating to resolve.  When inheriting from a Windows Forms Control class, always be sure to override both the OnPaint method and OnPaintBackground.  Even if you do not wish to paint a background or your control simply doesn’t have one, you must override this method.  While technically overriding the OnPaintBackground method is optional, the default is to paint over the entire control with a solid white, filled rectangle.  When attempting to do off-screen buffering, this will cause flickering!  The easiest solution is to override OnPaintBackground with an empty method:

protected override void OnPaintBackground(PaintEventArgs e){ }

This way, the previously drawn image will be left on the screen until it is overwritten by your off-screen drawn image.

Outside of this very annoying “gotcha”, off-screen buffering for a windows form control is very simple.  I’ve gone ahead and outlined the basics here:

// We need a bitmap variable to store our image
Bitmap offscreen; //The actual creation of the Bitmap is best placed in the OnResize method of your form control.  This way you will always have a correctly sized drawing area no matter what is done to the control.

protected override void OnResize(EventArgs e)
{
  this.offscreen = new Bitmap(this.ClientSize.Width - 1, this.ClientSize.Height - 2);
}
protected override void OnPaint(PaintEventArgs e) {   //In the OnPaint method of your control, you want to create a new Graphics variable to which you will do all of your GDI drawing.
  Graphics gOffscreen = Graphics.FromImage(this.offscreen);
  gOffscreen.FillRectangle(new SolidBrush(this.BackColor), this.ClientRectangle);
  // Clear the drawing area
  gOffscreen.DrawString("Test", Font, new SolidBrush(Color.Blue), ClientRectangle); // Draw your control graphics

// Once you are finished drawing to the off-screen image, simply draw your bitmap image to the screen and then dispose the graphic.

  e.Graphics.DrawImage(this.offscreen, 1, 1);
  gOffscreen.Dispose();
}

The Program
My current project is writing an IRC client for the Windows Mobile platform.  While the programming for the IRC client in and of its self is fairly straightforward (especially with the help of the .NET Compact Framework), the user interface is less so.

One feature that I would like to support is text formatting.  If you think of a standard IRC client, it’s really no more than a giant multi-line text box of people’s messages.  Most contain formatting and color options for special events such as server messages, channel joins, emotes, URLs, etc.  I wish to duplicate this functionality in my program for Windows Mobile.  If this were a standard C#.NET application, the solution for user interface would be very simple thanks to the provided RichTextBox windows form control.  But as fate would have it, the Compact Framework only has support for the normal TextBox control.

The Research
Posed with this dilemma, I began looking for alternative solutions.  One site that came up through searches was OpenNETCF (http://www.opennetcf.com/) which provides a number of libraries for extending the (lacking) functionality of the normal .NET CF.  One particular library which they provide has generated a lot of interest on various MSDN forums for Windows Mobile development; the Mobile Ink Library (http://www.opennetcf.com/Default.aspx?tabid=262).  This library is actually nothing more than a .NET wrapper for Microsoft’s RichInk Control (http://msdn.microsoft.com/en-us/library/bb160726.aspx).

The RichInk Control is really more or less the Windows Mobile replacement for a RichTextBox.  Rich Text Formatted (RTF) data can be streamed into and out of the control.  It also has pen support and many other features.  Unfortunately, this control is really only accessible when writing a C++ program as there is no .NET interface for it.  That’s where the aforementioned Mobile Ink Library comes in.  However, if you do not wish to use a 3rd party library, the RichInk control can be invoked from C# code.  The only trouble is that you will end up with some rather unsightly code and plenty of calls to SendMessage for dealing with control input/output and more DllImport statements than you can shake a stick at.  However, if you would like to attempt this approach, there is a post in the MSDN forums with some helpful code samples (http://social.msdn.microsoft.com/Forums/en-US/vssmartdevicesvbcs/thread/2dc97d42-4f97-4510-8e04-9b6a42b74184/).

All things considered, I ended up reaching the conclusion that the RichInk control actually hadtoo many features for what I really wanted to do.  To this end, I began looking at how I could either modify the existing TextBox control or write my own TextBox from scratch.

Be Warned!
There is an error in the MSDN documentation.  While I thought for a while that I could simply override the OnPaint method of the standard TextBox control, just as you can in the regular framework (http://msdn.microsoft.com/en-us/library/system.windows.forms.control.onpaint.aspx).  Even though the documentation does list Windows Mobile as a supported platform for this method, the OnPaint method is never called!  So, if you write a custom control which inherits from TextBox and override the OnPaint method to do any custom drawing, you will see no changes what-so-ever.  The OnPaint method is simply never invoked under any circumstances with the .NET compact framework.  This does include the most recent version, 3.5.

What’s next
After so much research and so many paths leading to dead ends, I have finally decided to write my own TextBox control from scratch.  While this does pose a bit of a challenge, it is not unreasonable.  My custom text box does not have to support any text editing.  Only display and selection support is needed.  With, of course, the display supporting various fonts, colors, etc.  I will be making a few posts in the near future as I develop this control, citing various resources which I have found useful.  Hopefully this content will be able to help someone else in the future with their own Windows Mobile application.

This is actually related to a work project which I finished about a month ago.  The situation was relatively straight-forward.  A program was needed to interact with a device via serial communication.  It needed to be able to send commands and log streaming output from the device.  Basically what I needed was a simplified HyperTerminal application with more dynamic logging options.  The reason why HyperTerminal was insufficient in the first place was that I needed to be able to generate and log timestamps along with the rest of the information streamed from the device.

Initially, I spent about an hour searching around the net for an existing program that would be able to do this job for me.  Unfortunately the only resources that I was able to find were payware applications ranging from $50 to $200.  Paying for one of these existing applications was not an option, so I resolved to writing my own from scratch.

Like most software development projects, I needed a working solution two hours ago.  My initial thought was to attempt something with C# as it was a higher level language and I knew that .NET had some serial interface classes built in.  Unfortunately I did not have access to a computer with Visual Studio already installed on it.  While the express edition would have worked just fine, it would have taken at least another hour or two to download and install.  Fortunately, I came up with another idea that I thought might be worth looking into.

My experience with python in the past has been fairly limited, but I was familiar enough with it to know that I should be able to come up with a working solution very quickly.  Of course it wasn’t until I started programming that I realized Python has no native libraries for dealing with serial port communication.  Luckily, with a bit of googling, I discovered an open source python library called pySerial:

http://pyserial.sourceforge.net/

While the documentation is a bit lacking, the provided example applications more than make up for it.

http://pyserial.sourceforge.net/examples.html#miniterm

All in all, python is a great tool for cobbling together what should be a simple application at the last minute.  Just be sure that it has built-in or open source 3rd party libraries for whatever it is you want to do before committing to it.