World’s Cheapest SCADA Device: Conclusions

For the sake of brevity, I’m going to refer to this type of device as a General Purpose SCADA Device, i.e. GPSD.  No, wait, that’s already taken.  How about “GPD2”, for General Purpose Digital Device (the “2” is silent)?

What I learned:

  • OMFG this is painful.  Hooking up a random new device is hard.  My hat is off to the EIE team.
  • Having control of both ends of the conversation (app-level protocol) makes things much easier.
  • As a software person, did I mention the pain of talking to the real world?  Gahh….really, why is it this hard?
  • It was fun.  Honestly, I’m someone who enjoys their work, but this was just ridiculously interesting.  Jeff says it never goes away, that excitement when software makes something happen in the Real World.
  • Cheap.  This was insanely cheap.  I was using 2 out of 16 general-purpose I/O lines.  Parallax actually sells embeddable OEM versions of some of their more capable chips.  Their latest product is a single-chip 8-core machine programmable using real languages and tools.
  • Despite the feeling of being frustrated, the reality is that the majority of my time was spent learning about CygNet and relearning some elementary electronics.  The vast majority of time was spent doing two things:
    • Designing the protocol – my original design just streamed data out the serial port *and* responded to commands.  Since the original device was a beeping, science-fictiony kind of meter, I changed it to a request-response architecture (at Derek’s urging.)
    • Learning about the EIE stuff.  Doug was out when I initially tried to get the GenRawMsg working, but it only took Sheri (and me) a couple of hours to get it working.  I would also say that this is a testament to the fundamental sanity of the architecture
  • The market for GPD technology is insane.  Pricing makes no sense.  There are hundreds, maybe thousands of supplier of hardware, software, and services.  It’s like walking down 47th St. in New York, where all the random grey-market electronics vendors ply their trade.  Everyone sells everything for less than everyone else, and yet people end up spending lots of money.
    I’m assuming that there hasn’t been a shake-out because requirements/needs for IRL (In Real Life) data have been few and far between.  I would expect this market to follow the same patterns of all digital tech markets to date.  In the meantime, the more we know, the more likely it is that we’ll be able to pick some of the protocol/design/hardware winners in advance.
  • Some of the CygNet UIs are hard to use.  I can’t decide if they need to be this hard to use.  I’m thinking specifically of the DEID/datagroup/UIS commands.  Given the extremely high specificity of given devices, and the rarity of hooking up brand new devices, I’d be inclined to give this a pass, especially if we development some generic EIEs that allow protocol definition and specifications using simpler (and as yet undesigned and unimplemented) tools.

I haven’t used existing PLC/RTU’s except the Fisher 503 in training.  As a result, my perspective on SCADA is informed primarily by my experience as a software designer and developer.  It looks to me as though this is another one of those “push them off the cliff” situations where existing manufacturers feel that they’re in no danger of being supplanted by “toy” systems.  This is the exact same argument applied to the telephone, computers, personal computers, automobiles, etc.  I haven’t seen anything to suggest that this is any different, but I’m open to the idea.

There are dozens of protocols, at the very least.  Every time I look one up, I end up diving into a rathole that invariably leads to 1980 and a statement like, “Company Foo, finding existing technologies inadequate to the task of manufacturing anodized, left-handed smoke turners, decided to team up with 4 homeless electrical engineers and design a protocol optimized for that task.”  The result is the hodge-podge of protocols (hardware, software, application) currently in use.  I’m not saying, by any stretch of the imagination, that these are unnecessary, or bad, just that the sheer number is terrifying to someone used to markets where there tend to be only a few dominant architectures.

What I have seen of the protocols and capabilities suggests strongly that these dedicated devices have a very long tail due to their industrial construction and simple nature.  It’s not like they’re going away, certainly not in our market while petroleum costs are high.

At the same time, markets without existing or significant SCADA penetration will be more likely to turn to GPDs based on their lower cost to acquire and program.  More importantly, the utter lack of flexibility in existing designs will push penny-pinching small companies with novel SCADA requirements in the direction of software-based systems, rather than systems limited/constrained by decades-old hardware designs.


to be answered at a Research Directions meeting.  My responses are in blue.

  • Should I continue to develop this device so that it can serve as a complete, end-to-end demonstration?
    • Yes. In fact, I’d like to switch to a more sophisticated device.  Note that the cost increase for hardware might be as high (hah!) as $120 for a Cubloc device (w/training) that provides “real” features found in existing RTU/PLC, e.g. ladder logic, large numbers of I/O ports, MODBUS, and floating point calculations.
    • – products
    • – starter kits
  • Should we develop a “generic” EIE that supports cry-outs in addition to raw messages in both variable- and fixed-length versions?
    • If we believe (and I do) that the future of SCADA involves devices considerably smarter and cheaper than traditional PLC/RTUs, then we’re going to want to be ahead of the curve in understanding how to best make use of these “generic” SCADA (or anything) devices.
  • What about developing some “kits” that serve to demonstrate the wide applicability of SCADA-originated data?  E.g.  an environmental-monitoring “pod” containing sensors for temperature, GIS info, water quality, etc.
    • This could be done as a side-effect of the prior two items.  I know some environmental scientists who might interested in collaborating on the design and use of devices like this.
  • Do we want to begin pursuing publication opportunities in industry trade rags?  I have a hard time believing that any rational editor would pass up an article entitled, “The World’s Cheapest SCADA Device,” for instance.
    • I think we should, but I don’t know anyone in the right industries, I think.  I’ll be happy to write stuff.

Cross-tab data visualisation

Parallel Sets

Parallel Sets (ParSets) is a visualization application for categorical data, like census and survey data, inventory, and many other kinds of data that can be summed up in a cross-tabulation. ParSets provide a simple, interactive way to explore and analyze such data.

123 Main Street, Anytown, USA – now with pictures


This is the most interesting (to me, that is) free map application I’ve seen.  You’re looking at my neighborhood in Appleton in several ways, simultaneously.

I found it because the city of Appleton links to it from their "My Appleton" site:  which is sitting on one of the canonical ESRI GIS systems (which I know only because I saw upgrade costs in the city budget docs.)


I’m updating Cygnipede and have been spending time looking at how this sort of data can be used.

Aging workforce


The number of residents in North Dakota age 65 or older is forecasted to grow by 58 percent in the next two decades when it would represent 23 percent of the state’s population, according to projections from the North Dakota State Data Center at North Dakota State University. The number of Minnesota residents turning age 62 is also expected to jump by 30 percent in the next year.

North Dakota’s oil and gas industry, coal mining and power generation facilities are projected to lose about half of their workforce due to retirement in the next decade, according to Dave Farnsworth, power generation and engineering leader at Great River Energy’s Coal Creek Station near Underwood, ND.

Invensys virtual discovers VW-based training

I’m not sure how I feel about this.  On the one hand, there are few people with more experience at this sort of thing than I have.  On the other hand, I’d have to say that we’ve found that VW/VR-based training seems to be suitable for tasks which closely mirror the training environment, which is to say that tasks which will be carried out using purely-digital UIs.

The missing piece, or rather an important missing piece seems to be that VW/VR-based status displays could offer some interesting enhancements to traditional "steam gauge" technology.  NOAA, for instance, has a weather map of the United States that they’ve implemented in SecondLife.  It uses 3D objects to show things like storm systems, etc.  And,since the display is 100+ feet across (according to your avatar’s size), you can "fly" through US weather, lending a perspective that couldn’t be seen in any other fashion. 

Would having a pipeline simulated in this fashion be helpful? 


Using Java to interact with WCF (CygNet Enterprise Gateway And Integration Services (CEGAIS))

Posted without any commentary since I mostly need this for future reference. -Lynne



Sun’s Project Tango


Print-friendly VersionPrint-friendly Version

By Harold Carr, June 2006

Duke Tango Sun’s Java Web Services engineers are working with Microsoft’s Windows Communication Foundation (WCF) engineers to ensure interoperability of enterprise features, such as security, reliable messaging, and atomic transactions. At Sun, we call this work "Project Tango" and like to think of Duke doing the dance. This is accomplished by implementing a number of WS-* specifications and holding plugfests where Sun and Microsoft engineers work face-to-face testing our implementations of those specifications.

From a functional point-of-view, the main interoperable features enabled by Project Tango are:

  • Bootstrapping communication
  • Optimizing communication
  • Enabling reliability
  • Enabling atomic transactions
  • Securing communication

Project Tango is developed as open-source as part of the Glassfish Community, and it is delivered integrated into the latest Web Services stack there. Sun plans to release Web Services Interoperability Technology (WSIT) products based on this stack. The WSIT functionality is delivered as plugins to Sun’s JAX-WS implementation, and the implementation code is here.

The rest of this article gives an overview of some of the features enabled by Project Tango.

Bootstrapping Communication

When you want to use a web service, you must first find a URL to the service, then build consumer-side artifacts to communicate with the service. Finding the service is typically done either out-of-band or by looking up services in a registry. In Project Tango, once a service URL has been obtained, the URL is passed to wsimport. wsimport retrieves the metadata describing the service (for example, WSDL) and creates consumer proxies used to communicate with the service.

Project Tango relies on a newer version of JAX-WS‘s wsimport to use WS-Transfer and WS-MetadataExchange to obtain a service’s WSDL from a WCF-based service. These specifications are used as an agreement between Sun and Microsoft on how to transfer the metadata, and don’t need to concern the developer. All this is hidden under the covers by wsimport.

Optimizing Communication

Project Tango provides two optimizations for communication between web service consumers and web service providers:

  • Optimizing the wire format of SOAP messages.
  • Security optimization.

The wire format is optimized by MTOM and XOP (built into the underlying JAX-WS platform). These specifications enable optimized binary encodings of XML content within SOAP 1.2 payloads. XML structure is preserved in a text-based "XML Information Set" manifest, while enabling the document’s contents to be serialized to any binary encoding. These specifications enable encoding of XML content as MIME body parts and encapsulation of those parts — along with the associated XML Information Set manifest — within SOAP 1.2 envelopes. The specifications also describe how to encapsulate binary-encoded XML body parts directly within HTTP packets. The main point is that the size of the XML is reduced resulting in more optimal communication.

Security is optimized by implementing the WS-SecureConversation specification that enables a consumer and provider to establish a shared security context when a multiple message exchange is first initiated. Subsequent messages use derived session keys that increase the overall security while reducing the security processing overhead for each message.

Once again, a developer does not need to be concerned with the specifications. Project Tango uses these specifications to enable optimizations that work with WCF-based consumers and providers. When creating a Project Tango-based provider, enabling these optimizations is simply a checkbox feature when using the Project Tango module for the NetBeans IDE.

When consuming a web service, nothing special needs to be done – the optimizations are described in the service’s metadata (for example, WSDL) and are handled automatically when creating the artifacts with wsimport.

Enabling Reliability

Reliability, in Project Tango, ensures that the underlying system is able to recover from failures caused by messages that are lost or misordered in transit. Recovery is handled by the underlying system without involvement of consumer or provider application code.

Interoperable reliability is achieved by implementing the WS-ReliableMessaging specification. Turning on reliability, in Project Tango, when developing a web service, is simply a checkbox in a reliability panel in NetBeans as shown in the following screenshot.

Reliable Messaging Screenshot

Reliable Messaging Screenshot
Click here for a larger image.

Enabling Atomic Transactions

Atomic transactions are a well-known means to ensure that all operations within a transaction boundary succeed. If even one operation fails then all other operations are aborted. Transactional support has been available in Java Enterprise Edition for a number of years on IIOP invocations. Project Tango enables transactional support for web services.

Interoperable atomic transactions are enabled, in Project Tango, by implementing the WS-Coordination and WS-AtomicTransactions specifications.

Securing Communication

Web services, until now, have relied on transport-based security such as SSL to provide point-to-point security. Project Tango implements the WS-Security specification to provide interoperable message content integrity and confidentiality, even in the presence of intermediaries. In other words, WS-Security does not rely on transport-based security. WS-Security is a security option provided by Project Tango besides existing transport security which still may be used. Project Tango also provides an implementation of the WS-Trust specification as a means for issuing, renewing, and validating security tokens used by WS-Security, and to establish and broker trust relationships.

Other "Invisible" Specifications

A developer using WSIT products from Project Tango does not need to be knowledgeable of the above specifications nor to even be aware of their existence. The specifications are design documents that enable Project Tango and WCF to interoperate by implementing those specifications. In addition to the specifications listed above, interoperability with WCF is also using WS-Addressing (normalized web service addresses, enabling multiple transports to be used, besides HTTP), WS-Policy (configuration), WS-SecurityPolicy (policies that describe how messages are secured).

NetBeans Support for Project Tango

There is a NetBeans module that simplifies writing and configuring a web service application to use the new features enabled by Project Tango. A developer writes a web service application using JAX-WS. The developer then configures reliability, transactions and/or security by selecting appropriate settings. Here is an example security screen.

Security Screenshot

Security Screenshot
Click here for a larger image.

Interoperability Testing

In addition to daily interoperability testing between Sun sites and Microsoft sites, Sun engineers worked with Microsoft engineers in Redmond in November 2005 and again in March 2006 at interoperability "plugfests." This focused interoperability testing ensures that WCF-based products and Project Tango-based products will seamlessly work together out-of-the-box. Arun Gupta’s blog is a good place to go to find more details about our work at the plugfests.

For More Information

  • You can pick up the latest Web Services stack which includes the WSIT technology at
  • You can also contribute to the development of the source code for Project Tango at and to the base JAX-WS infrastructure at Both are part of the Glassfish community.
  • TheAquarium is a good place to keep up-to-date on the Glassfish community.
  • The author of this article, Harold Carr, maintains a blog with entries pertaining to Project Tango.

The problem with BlackBerrys (part N)

Wow, am I annoyed.  A whole day just got wasted when two different BlackBerrys (8330, Storm) apparently died.  So I spent hours looking for a possible solution, finally deciding for no good reason to plug them into a different machine.

Guess what?  They *then* decided to boot.  Plugged them back into the first machine, and they’re fine.  And no, it wasn’t the USB ports, because I tried other devices in the same ports and they worked fine.

I think this was yet another example of the BlackBerry Desktop Manager just not working, which happens a *lot*.  They’re working now, which is a relief, since I thought I’d bricked them.