Quantum of SCADA

So…yeah…today sucked.  Is it possible to have negative productivity?  After running off the track (not too far) yesterday, Jeff pointed out the correct tracks far off on the horizon and I trudged back, shoved the consist back on the rails, and got re-started.  Of the dozens of lines of unnecessary (but not bad) code, I only had to undo a couple.  Monday, Tuesday, and Wednesday I made great progress.

Today?

Not so much.  Write a line of code, delete it, or maybe two.  Follow a trail of references, definitions, and implementations.  The saving grace for me is that our code base doesn’t suck, and wasn’t written by people who were stupid or stupidly-clever.  It’s well-organized, and mostly plain C (aka C+), and the code doesn’t rely on side-effects.

I’m in this weird position of knowing a lot about individual leaves on the CygNet tree, and I totally get the trunk, and am passing familiar with several branches. Beyond that, well, I’m not quite in “and then a miracle happens!” territory, but some of the things I think I know aren’t really so, and it means it’s sometimes difficult to know WHEN TO STOP MOVING IN A PARTICULAR DIRECTION.  Ahem.

Today was adding a new registry item for CygNet service/client startup, “IpVersion”, which all future (assuming we do v6) clients and services will have to check on start up so that they know version of IP to try and use.  Argh.  The less said, the better. 

What does this have to do with quantum?  Nothing.  I was just thinking what it would be like to build a SCADA system where points didn’t have current values, per se, until you actually tried to use them.  Instead of recording a value, you’d record a lambda function that expressed the possible quantum superpositions of the point involved.  I think I have that right.  We have services that are like that now, but I was thinking about what happens when you literally have to record things that don’t have a current value, just a function describing possible values.

OTOH, I’d hate to be in Support’s shoes and have to say to a customer, “Well, first we need to collapse your system to see what’s going on…”

BTW, if you’re not familiar with the Lambda calculus, Alonzo Church, and Alan Turing, I strongly urge you to read up on it.  It changed the way I think about software in fundamental and useful fashion.  http://en.wikipedia.org/wiki/Lambda_calculus

Advertisements

Cygnipede client-side persistence

As of v2, Cygnipede on the client (BlackBerry) is two applications:

  • UI – a midlet, rather than a RIM applet
  • Listener daemon – sits in the background, listens for location updates, and transmits results to one-or-more CygNet EIS gateways

Today I’m hooking up (and extending the old classes) the persistent store code.  We’ll see how that works. 

<time passes>  Late afternoon…

I. Am so. Angry.  That I can barely describe it.  I have spent more than half of my day trying to figure out why *both* BBs just stopped communicating with the BBDevMgr.  I’ve rebooted 3-4 times, maybe more. Reinstalled the BB desktop software as many/more times.  Now they’re working, for no reason I can understand.

For reference, here’s what I did.

  1. burnt offering to Atropos, cutter of cords
  2. Uninstall Desktop Manager
  3. Shut the machine down
  4. Unplug all USB devices (except keyboard/mouse)
  5. Reboot and disable all protective software, turn off all services that might be listening for a media device to be plugged in, etc.
  6. Reinstall Desktop Manager
  7. repeat step 5
  8. Plug in the 8330 and open Device Manager.  It’s there, with a notice that the driver might be corrupt.
  9. Without unplugging the device uninstall and delete the driver.
  10. Unplug the device.  Count to 100 by primes.  Plug it back in
  11. Allow it to install the current drivers
  12. Watch as “Blackberry Smartphone” appears magically in the DevMgr
  13. write angry blog post about being angry

Searching Google for “blackberry not recognized” is a real eye-opener, because there are too many results, and none of the results (really, I checked, none of them) have answers from RIM (or anyone else) that’s helpful.  They all boil down to, “do things randomly.  sacrifice something.  dance in the moon.” 

They were recognized by the operating system, which is the super amazingly annoying part.  Device manager said, though, “There’s a problem and maybe the drivers are corrupt.”  What?  Corrupt?  No. That doesn’t happen, because if it did, the disk manager would tell me I was getting disk errors.  That happened in, oh, I dunno, 1985.  Not now.

This whole Cygnipede2 experience has been insanely frustrating, to the point of tears.  Everyone has been on board, very helpful and interested, offering lots of useful suggestions.  Derek and Jeff have been in on feature decisions, and I feel as though a lot of this has been torpedoed because I’ve probably spent 1/3-1/2 of my core development time dealing with bullshit issues like this.

Hey, RIM!  Don’t you get it?  *THIS IS WHY THE iPhone IS WINNING!!*  Yeah, because their platform works, and even though I don’t like their development platform, it would be novel to have one that, say, worked more than 50% of the time.  I’m sitting here thinking to myself, “Did I start too late?  Am I stupid?  Am I too lazy?  Did I do the wrong thing?”  The answer has to be “probably not”, because I have never in my life said, “50% of my development schedule is going to be devoted to reinstalling and rebooting.”  I’ve been doing this for 30 years and I can say without exaggeration that this has been one of the worst software development experiences of my life, owing solely to what I can only see as massive incompetence on the part of the vendors involved.

Another problem for the past month was the Desktop Manager deciding, apparently arbitrarily, that “This application is incompatible with your device,” on what appeared literally to be a random basis.  How do I know this was nonsense?  Because you can use javaloader to install the applications and they work fine.

So…where was I, 8 hours ago?  Oh, right, persistence.  And Yong’s sent out an installable gateway, which is what I think I’ll do next, because I’m about 10 minutes from pitching these damn things into the backyard for the squirrels.

To paraphrase,

“Continuing to develop software is a clear example of the triumph of hope over experience.”

Cygnipede After Action Review

~ or ~ 
Development Considerations Reconsidered
~ an Adventure in Too Many Parts ~

AAR

Well, calling it an AAR would certainly be optimistic, because I was about two weeks behind schedule when it came time to actually use Cygnipede v2 (C2).  Why?  Well, you know the phrase, “a good worker doesn’t blame her tools”?  I’m about to do some of that.

Short version: 

  • I missed the ship date (City To The Sea)
  • Learned a *lot* about the [lack of] stability in BB dev tools and platforms
  • Am continuing to upgrade C1 to C2, especially as regards the CygNet Enterprise Gateway

There is no question in my mind that, left to my own devices, I will slip schedules in order to add “one last feature”.  In that regard, I’m no different than most software developers.  In this case, though, I didn’t get to the features we wanted, and a big part of it was some mistaken assumptions on my part about both the execution environment (i.e. the phones) and the development tools.

Given that the BlackBerry is a mature platform as mobile platforms go, I assumed (implicitly) that time would further stabilize both of these, cleverly failing to note an entire year of errors like, “Cygnipede won’t run,” and “Cygnipede can’t find the GPS,” from all and sundry, including YC.  Familiarity breeds contempt, so I should have noticed I’d managed to generate an entire flock.

Our initial goals

  • decreased power consumption
  • more (ok, *any) status information for the mobile user
  • work out issues with “push” installations
  • demo booth at City To The Sea
  • and the late addition – use the new Cygnipede Enterprise Gateway

Background

The core of the C1 application is a markedly modified version of one of a sample GPS application from the SDK.  As demo applications go, it was ok, but was both too complicated and not really aimed at our purposes.  It was monolithic and multi-threaded, and this on a platform that does not reward complex architecture.

Development on the BB can be done in three ways:

  • BlackBerry JDE (Java Development Environment) from RIM – essentially this is the simplest possible IDE you could have and still do work.
  • Eclipse with RIM plug-in – open-source IDE from IBM with an extensive set of tools
  • NetBeans from SUN + JME (Java Mobile) add-ins – notable because it’s the only one of the three that offers a drag-n-drop GUI for UI design, although only for applets and not straight-up Java applications.

Problems Encountered

Mistakes

  • Trying to make the client UI a MIDLET.  The limitations in control layout alone are enough to make me say, “Never again.”  I thought having a GUI designer/code-gen tool would save time, but I was wrong.
  • In general, trying too many things.  Next time:  break one thing at a time.
  • Let myself get distracted.  In such a chaotic environment, i.e. mobile development, there are lots and lots of really interesting things.  Just concentrate on making one phone do one thing with one server at a time.
  • I can’t decide if this was a mistake, but using a sample application as the basis might have been wrong.  I spent time trying to make someone else’s code do things it wasn’t really designed for.  If I’d spent the same time writing two separate applications from the ground up, I imagine I wouldn’t have spent any more time, and would have ended up with something that involved fewer kluges.

Devices

  • The Storm caused the most problems – I ended up having to wipe mine every few days when it would get to the point that attempting to update via the BB-DesktopManager would result in a ‘fatal error’ message (in BB-DM, with no further explanation).
        Eventually I figured out that the DM was trying to update the OS and update applications.  Once I made it do those tasks separately, it got a little better.
  • “GPS services are not available on this device” – eventually diagnosed as being the result of running an application that tries to use location services with an external GPS, then fails and switches the entire device to the internal GPS without telling the user.  Next time Cygnipede starts up, this happens.  Pull the battery and reboot.  Switch back to the external GPS.  Now it will work.
  • Simulators don’t – in theory it’s faster to use the simulators, but even the RIM JDE has problems connecting to them and I discovered that at least in the case of the 8330 (Curve), the emulator and actual device behave different, even with both of them running the same firmware version.  Things that worked on the emulator wouldn’t work on the physical device, although the reverse didn’t seem to be true.  Had to abandon using the emulators due to the uncertainty surrounding reliability. 
  • Running the battery completely down results in unpredictable behavior when the device reboots, although this is anecdotal and (I suspect) not actually the case.  The problem is that when problems stack in this fashion, the superstitious part of one’s brain tends to run a bit wild in the speculation department.

Development Tools

  • JavaDocs – While Visual Studio is at the point where there’s almost too much documentation, the documentation for the Java tools consist of:
    • An outline of the classes (API)
    • No examples to speak of along – this is just whining, frankly, because I’m used to having functional and non-trivial examples to work from, a la Visual Studio
    • Oddly, the IntelliSense-like auto-completion feature in Eclipse is arguably better than VS, because it allows what I find to be a more natural navigation to relevant docs/help.
  • BlackBerry Desktop Manager – as suggested, I used v5, but it was no more stable than v4.7.  The typical problem was that after re-plugging a device about 10 times, the BB Device Manager (different software from the Desktop Manager), would stop being able to see it.  The Windows Device Manager would see it, though. 
        This would be bad enough, but apparently the charging circuit doesn’t work if the driver on the PC doesn’t see it, so to all appearances, the device batteries had failed, since plugging them in wouldn’t result in them charging.  The solution was to reboot the PC, remove the BB Desktop and Device Managers, reboot, install the BB Desktop and Device Managers, shutdown (power off), and then upon booting the devices would be visible.
        Lest you think this was a machine-specific problem, I was experiencing these on both a Dell XPS laptop (Windows 7) and my home-built dev box (Vista Ultimate).
  • In general, the RIM-recommended approach to problems seems to be, remove, reboot, and re-install software.  We are not amused.
  • Vast numbers of devices and versions – this isn’t anyone’s fault, it’s just that there are a dozen BBs, and a half-dozen carriers each with their own restrictions on the APIs.  Our solution was to just say, “If it can’t do v 4.5, we’re going to ignore it.”  We should probably actually say, “If it can’t do 4.7 or maybe 5.0, we’re not going to support it,” since these versions of the BB OS have reasonably comprehensive APIs that support the interesting parts of modern devices (e.g. accelerometers, faster location services, better UI suport).

Helpful Resources

  • Safari books subscription – can’t say enough good things about this, and not just for Java, but for *anything* having to do with digital technologies.  I was about to go out and buy a book about 3D lighting (for rendering) when I checked Safari on a whim.  OMG!  It was there, and I saved myself $40.
  • RIM BlackBerry discussion forum – almost any question one might have is likely to have been answered in some form
  • Eclipse – the only drawback is that the RIM plug-in requires an older version (3.2 – Ganymede)
  • Sun Java stuff

Architecture Solution

  • Split the application into 2 (or more) sub-applications:
  • User Interface – set options and view status
  • Location listener
  • Fix feeder (transmit feeds to CygNet and report on status)

The existing C1 has been updated to about v1.5, I’d say.  It includes the following changes

  • Client now transmits altitude, heading, speed, and a user-defined text message
  • Client shows extensive status information in a scrollable window

I’m currently working on posting to the Enterprise Gateway.

What I learned (I hope)

Once you have a working dev environment, DON’T CHANGE IT.  Since VM-Ware (in theory) supports USB connections, I’m considering building a VM just for this purpose.

On your dev box, try to avoid plugging any non-BB mobile device into your USB ports.  Plugging in my Windows Mobile phone seemed to be one of the things that would cause the BB Desktop Manager software to just wander off and never come back until a reboot.  Don’t plug in more than one BB at a time.

Use the ‘javaloader’ application instead of the BB-DM when possible, since it seemed more reliable.  Load the ‘.jad’ file instead of the ‘.cod’ file if you want it to appear in the menu somewhere

NetBeans, while superficially attractive due to the GUI builder offers no real advantages over Eclipse except for MIDLETs.

Mobile platforms are far more finicky about things being Just So.  I knew this, but had on some Happy Ears.

What about next year?

<bad words>

Not really.  It was enjoyable, and we’ve learned some interesting things about…mmm…not limitations in CygNet so much as possible way to increase the richness of the architecture.  I’m specifically thinking of the fact that GPS fixes are compound entities which are nearly always accessed en toto, rather than in parts.  A latitude or a timestamp aren’t generally useful, except as part of the whole piece of information.

As it stands, we represent the fix as several different points of the form

  • PH0001112222_(LAT|LON|TSR|ALT|SPD|HDG)

That’s not a problem, but I’m thinking about whether it would be useful for fixes to either have their own service, or for points to be able to define a collection of attributes as representing a “type”, in the computer science sense.  Basically a point that’s a collection of points.  It’s all too easy for this approach to cascade until you find yourself with an actual OODB, with bad performance and complexity too high for anyone to actually use. 

I’m not suggesting a general type system, just that we can currently only represent scalars, and in only canonical mathematical types.  I know we do composites, but I’m specifically referring to points of enduring business interest, to quote a colleague.  I’m specifically referring to tuples as a possible way to create 2nd-generation SCADA objects, and GPS fixes are an exquisitely good example, in my opinion, of something we don’t/can’t currently do well.

 


I just heard that 3 people died during the Detroit Half-Marathon and found myself wondering if Cygnipede could help.  A Bluetooth-enabled health monitor broadcasting runner stats might have allowed a centralized system to identify people who were approaching some kind of crisis point.  I need to go talk to some doctors about this.

Summary

Cygnipede walks on, and I’m hoping we find some more uses for it.  My current plan is to keep working on it as a very-low priority background task, unlike last year when it got dropped until we needed it again.

Dueling include files

I was working on SupportNet when I realized I needed a function to tell me whether a string buffer contained a syntactically valid IPv4 *OR* IPv6 address.  First thought was some string checking, but there are a *lot* of valid ways to write IP addresses, so that went out the door.  I remembered discussing this with Mike W and we’d decided that using the inet_pton & inet_ntop functions were probably the way to go.

No problem.  Except…I can’t find them.  MSDN online has documentation for them, but the local installation doesn’t.  Worse, as it turns out, I have the Windows Device Driver Kit (SDK) installed, because it had an import library for some obscure RTL functions I’d tried, and I never removed it.  Bad idea, because there seem to be some overlapping definitions (macros).  I uninstalled it and *those* problems went away, but I still couldn’t seem to get VS to find the inet_* functions.

Net-net:  CygNet is built to run on Windows XP and later, but IPv6 isn’t supported very well there, so we’ve agreed to say “IPv6 is for Vista and later”.  After dealing with the “ideal” location for the winsock2 includes, and the Windows version updates, I’ve updated CygNetStdAfx.net as follows, and this seems to work.

I also ended up having to delete every NCB (Intellisense database) in the CygnetSource tree in order to get the inet_* functions (and other new winsock2 functions) to appear in auto-completion lists.  Very annoying.

 

CygNetStdAfx.h

//
//NOTE: IPv6 – trying to support v6 on XP would be a nightmare, and futile, and a PITA, and other bad things, so I’m going to require Vista or better. -Lynne
//

//#define WINVER         0x0501  // Windows XP+
//#define _WIN32_WINNT   0x0501  // Windows XP+
//#define _WIN32_WINDOWS 0x0501  // Windows XP+
#define WINVER NTDDI_VISTA  //  Vista = 0x0600
#define _WIN32_WINNT   WINVER  // Vista
#define _WIN32_WINDOWS WINVER

#define _WIN32_IE      0x0600  // IE 6.0+