Devices That Need Timestamping

Short version:

Wireless webcam feeding images to BSS sends them with wrong timestamp because device enables normal functioning before getting a reliable NNTP update.  If you’re using “smart” network-based devices, make sure that they’re capable of always providing data with reliable timestamps.

 

Long version (ranting included for no extra charge)

I’m doing a project that involves importing images from a motion-activated camera into the BSS (Blog Storage Service).  Because I had one lying around, I used a http://www.linksysbycisco.com/US/en/products/WVC54GCA wireless, self-contained webcam.

Let me say, for the record, that this is a very poorly implemented camera, something that is not helped by Linksys’ refusal to provide timely firmware updates.  E.g.: If you use a dynamic DNS service like http://www.dyndns.com/, presumably you’re doing so because you want to, y’know, actually use it, maybe?

WRONG!

Why how come?  Because Linksys knows more than you about…INTARWEBS! They have a MASTERS DEGREE, in teh INTARWEBS.  As such, they know that your preferences are merely the pathetic dribblings of a puny mind, and in order to make you life better, you will be using TZO.COM for your DNS redirection service.  “Who?” I hear you thinking.  “Who?” say lots of users on the LinkSys message boards.  Yeah, “who?” exactly.  Never mind that DynDNS (and others) are around, and that you might have existing accounts and reasons for using them.  No.  You will use the Precious, the shiny-shiny Precioussss…..yessss….

Sorry….just went to Middle Earth there, just for a second.  The point, and there is one, I assure you, is that this device, when the power is removed, forgets a critical bit of data: the time and date.  Yep.  $.25 for a battery was too much for the product design geniuses, apparently.

“Big deal,” you say.  Doesn’t the device access an NNTP service during boot, as all other devices in Known Space do?  You’d think that, wouldn’t you, with your pointy little brain, but guess what?  You’d be, that’s right,

ALMOST WRONG!

There is a “Sync with PC” in the administrative interface, but no settings to allow you to specify a specific xNTP server/stratum/protocol.  And, since the device is never, at any time, connected to a PC, the actual functioning of this button, when it’s not mysteriously grayed out, is a Mystery of Science.  Presumably it’s actually using one, because at some point it seems to know the correct(ish) time, but you’d have to do protocol analysis to find out.  No settings for this, either.

All of this would simply be annoying, were it not for the piece de resistance: the device begins sending data BEFORE THE SYSTEM CLOCK HAS BEEN SET FROM A RELIABLE SOURCE.  “Ha-ha!” I hear you say.  Yup, pretty much.  In practice, this means that files uploaded by FTP (no, not SFTP, why would we want a secure connection?) have bad filenames, because the date+time is used for the filename.

“But Lynne, it’s just an inexpensive webcam, surely the designers…” NO! NO!  BE QUIET! BAD USER, NO PR0N FOR YOU!  I might be inclined to agree with you, except that I’ve used other networked webcams, in the same price range ($75+), and they’re just not this bad.

In reality, the device is too complex for a complete novice to use, and they’re presumably the target, given the price.  They’re not sophisticated enough for a user with a modicum of expertise, because they remove certain important choices. You know, I think we’ve all learned an important lesson, except the Windows Mobile UI design team:

Either make a thing an appliance, with a volume knob, or expose every single solitary setting so that experienced users can make it work correctly in any environment.

Conclusion

This specific device is only suitable for casual research projects.  I wouldn’t even use it at home.  My choice for inexpensive network cameras would have to be a D-Link unit, which even 6 years ago worked better.  We used them in retail establishments and spent about $350 each.  I’d link to their site, but http://d-link.com/products/category/?cid=37 is borked ATM.

At the high end of the scale are units like the http://www.axis.com/ one, designed for deployment in public and industrial environments, including hostile ones.  They also document and expose an API  Expect to pay 5-10 times as much as the D-Link models with similar functionality.

Oh, and the BSS import is moving along.

If you’d like to see the live feed from this unit:

(we’ve discovered that a lot of places won’t let this through their proxy/firewall/gateway, but it’s known to work from SLO, at least)

Advertisements

May you network in interesting times

Part and parcel of learning about IPv6 has been establishing an internal infrastructure capable of running it, and which must be readily reconfigurable in order to test common configurations.  Stephen H and I have spent dozens of hours trying various configurations of modern, common, small-network configurations as part of the research v6 testbed.

In no particular order, here are some things we’ve learned:

 

China  – For years we used to say to people who claimed to be interested in network security, “Listen, most Internet-based attacks are coming from address blocks allocated to China, and tracerouting would seem to indicate that there are thousands and thousands and millions of break-in attempts every day.” And…they’d look at me as though I’d shown them a mouthful of black beetles whilst wearing an aluminum-lined baseball helmet, so eventually I just quit talking about it. 

Mind you, this started in the early 1990s.  As I’ve been watching network traffic, I noticed it’s only gotten worse, so I’ve blocked basically all APNIC and Russian address blocks, which has significantly cut down on the random attack traffic I’ve seen.

Why should we care?  Because one big feature of v6 is that it [potentially] brings back the end-to-end architecture that is TCP/IP’s original primary design feature.  You won’t need NAT and everything will be hunky-dory, and or peachy keen, whatever those mean.  Unfortunately, one good side effect of NAT is security-through-inaccessibility, as in you can’t attack what you can’t get to.  With e-to-e v6, though, your hosts are potentially once again available for attack.  Because c6 hasn’t been widely tested (by crackers), it’s a good bet that many implementations will have a lot of security problems in their first few years of deployment.

Windows IP Helper Service – IPv6 transitional assistance service (“Provides tunnel connectivity using IPv6 transition technologies (6to4, ISATAP, Port Proxy, and Teredo), and IP-HTTPS. If this service is stopped, the computer will not have the enhanced connectivity benefits that these technologies offer.”)  While debugging some other network issues, I discovered that at least on one machine, having this service running was generating a lot of network activity to weird random IP addresses on the Internet.  It’s off for the moment until I can confirm that this is normal behavior.

 

DNS – Domain controllers with multiple NICs that are running DNS will not necessarily return the expected IP address.  The default setting is to round-robin-return the various NIC addresses that register themselves with DNS.  Our first thought was, “turn off round robin,” but that didn’t work, and then “don’t allow DNS registration,” and that didn’t work.

Current status:  not solved

Current “solution”: from Microsoft-> don’t run DCs with multiple NICs.  Seriously, this is their advice. 

PostVerta – Roman goddess of the past

sketch of a design for a UI to allow browsing alarms and associated images, or vice versa, via a variety of navigation mechanisms.  Designed for use by pointer only, if necessary.

The red arrows mean “previous/next critical (red) alarm” and the yellow mean “previous/next warning”.  You could extend this model to incorporate add’l graphic signifiers, to ease use on non-keyboard devices.

image

Quantum of SCADA

So…yeah…today sucked.  Is it possible to have negative productivity?  After running off the track (not too far) yesterday, Jeff pointed out the correct tracks far off on the horizon and I trudged back, shoved the consist back on the rails, and got re-started.  Of the dozens of lines of unnecessary (but not bad) code, I only had to undo a couple.  Monday, Tuesday, and Wednesday I made great progress.

Today?

Not so much.  Write a line of code, delete it, or maybe two.  Follow a trail of references, definitions, and implementations.  The saving grace for me is that our code base doesn’t suck, and wasn’t written by people who were stupid or stupidly-clever.  It’s well-organized, and mostly plain C (aka C+), and the code doesn’t rely on side-effects.

I’m in this weird position of knowing a lot about individual leaves on the CygNet tree, and I totally get the trunk, and am passing familiar with several branches. Beyond that, well, I’m not quite in “and then a miracle happens!” territory, but some of the things I think I know aren’t really so, and it means it’s sometimes difficult to know WHEN TO STOP MOVING IN A PARTICULAR DIRECTION.  Ahem.

Today was adding a new registry item for CygNet service/client startup, “IpVersion”, which all future (assuming we do v6) clients and services will have to check on start up so that they know version of IP to try and use.  Argh.  The less said, the better. 

What does this have to do with quantum?  Nothing.  I was just thinking what it would be like to build a SCADA system where points didn’t have current values, per se, until you actually tried to use them.  Instead of recording a value, you’d record a lambda function that expressed the possible quantum superpositions of the point involved.  I think I have that right.  We have services that are like that now, but I was thinking about what happens when you literally have to record things that don’t have a current value, just a function describing possible values.

OTOH, I’d hate to be in Support’s shoes and have to say to a customer, “Well, first we need to collapse your system to see what’s going on…”

BTW, if you’re not familiar with the Lambda calculus, Alonzo Church, and Alan Turing, I strongly urge you to read up on it.  It changed the way I think about software in fundamental and useful fashion.  http://en.wikipedia.org/wiki/Lambda_calculus

Computing in the real world, with giant spiders

This is exactly the sort of thing that many programmers wouldn’t really think about, that is, spiders interfering with data capture.  In this case, it’s just video from atop a hotel in Appleton, WI.  SCADA developers, on the other hand, would probably have already accounted for mammals, insects, and the occasional rain of frogs in their list of possible exceptions.

Giant Spider Menaces Appleton, Wisconsin

World’s cheapest SCADA device: source code

This is the source code (as of May 27, 2008) for the Parallax BasicStamp Recording light meter.  I apologize for the formatting, but in the exciting new world of Office 2007, there is *literally* no way to edit the HTML that Google or Help know about, so I can’t use the  <pre></pre> tag.

' {$STAMP BS2}
' {$PBASIC 2.5}

index VAR Nib

wStrength VAR Word
wCount VAR Word

soundTime VAR Word
bCmd VAR Byte

'  generic tmp variable
iTmp VAR Word
wTmp VAR Word

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'  '
' ''' Number of values to store in EEPROM
'  '
'
wMaxValuesToStore VAR Word
wMaxValuesToStore = 200
wCurrEntvalueLocation VAR Word
wStoredValuesCount VAR Word
wStoredValuescount = 0

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'  '
' ''' How long to wait (milliseconds) for command input before resuming the main loop
'  '
'
wCommandDelay VAR Word
wCommandDelay = 3000

OUTH = %00000000
DIRH = %11111111

UNUSED_DATA CON 65535
'
'  Commands that can be sent to the application
'
CMD_RESET CON 0
CMD_DUMP_LAST_VALUE CON 1
CMD_DUMP_ALL_VALUES CON 2
CMD_SLEEP CON 3
'  returns a fixed length buffer with known contents
CMD_DIAG CON 4
CMD_NOOP CON 254

bCmd = CMD_NOOP

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'   ''
' ''''''
' ''''''
'   ''
'
GOSUB Reset

DO
  GOSUB getRCTime
  GOSUB getCommand
  GOSUB Delay
  GOSUB updateDisplay
LOOP

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'
Reset:
  DEBUG "ResetStart",CR

  '  Make sure history memory ring buffer is set to a value that lets us later distinguish it from zero (0)
  DEBUG "Starting",CR
  DEBUG "Will save ", DEC wMaxValuesToStore, " values in EEPROM",CR

  DEBUG "Initializing history ring buffer",CR
  FOR iTmp = 0 TO ((wMaxValuesToStore -1) * 2) STEP 2
    WRITE iTmp, Word UNUSED_DATA
    'DEBUG DEC iTmp, "."
  NEXT
  DEBUG "Finished initializing history ring buffer",CR
  'DEBUG CLS, HOME
  wCount = 0
  wStrength = 0
  FOR iTmp = 0 TO ((wMaxValuesToStore -1) * 2) STEP 2
    WRITE iTmp, Word 0
  NEXT
  wStoredValuesCount = 0
  DEBUG "ResetFinish",CR
  RETURN

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'
getCommand:

  'DEBUG "cmd> "
'  SERIN 16,  84,getCommandBadData, 5000,getCommandTimeout, [WAITSTR cmdBuffer9]
  SERIN 16,  84, wCommandDelay, getCommandTimeout, [DEC1 bCmd]
  'SERIN 16,  84, [DEC bCmd]
  'DEBUG DEC bCmd
  GOSUB processCommand
  RETURN

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'
processCommand:
  'DEBUG "Got command: ", DEC bCmd, CR

  SELECT bCmd

    '  reset
    CASE CMD_RESET
      GOSUB Reset

    '  send current value
    CASE CMD_DUMP_LAST_VALUE
      DEBUG "", DEC5 wStrength, "", CR

    '  send last N values
    CASE CMD_DUMP_ALL_VALUES
'      DEBUG CLS, HOME
      DEBUG "" ,CR
      DEBUG TAB, "", DEC wMaxValuesToStore, "",CR
      DEBUG TAB, "", DEC wStoredValuesCount, "", CR
      DEBUG TAB, "", DEC (wCurrentValueLocation / 2), "",CR
      DEBUG TAB, "", DEC wCommandDelay, "", CR

'      FOR iTmp = 0 TO ((wMaxValuesToStore -1) * 2) STEP 2
      FOR iTmp = 0 TO ((wStoredValuesCount *2 ) + 1) STEP 2
        READ iTmp, Word wTmp
        IF wTmp = UNUSED_DATA THEN EXIT
        DEBUG TAB,"", CR
        DEBUG TAB, TAB,"", DEC iTmp, "", CR
        DEBUG TAB, "", CR
        wtmp = 0

      NEXT
      DEBUG "", CR

    CASE CMD_SLEEP
      DEBUG "Sleeping", CR
      SLEEP 1

    CASE CMD_DIAG
      DEBUG "deadbeef0",CR

    CASE CMD_NOOP
      'DEBUG DEC wCurrentValueLocation,"->",DEC wStrength,CR

      'DEBUG "Received NOOP command", CR
'      NAP 0
      DEBUG 0, CR

    CASE ELSE
      'DEBUG "Received unknown command", CR

    ENDSELECT

  RETURN

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'
getCommandBadData:
  'DEBUG "Got bad command", CR
  DEBUG "CommandError", CR
  RETURN

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'
getCommandTimeout:
  'DEBUG "No command received" ,CR
'     DEBUG DEC wCurrentValueLocation,"->",DEC wStrength,CR
      'DEBUG DEC wStrength,CR

  RETURN

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'
getRCTime:
  '  These next 3 lines are the actual application.  How long is it taking for a charged cap to drop to 0?
  HIGH 2   '  arbitrary short time to charge the cap (from the training example)
  PAUSE 3  '  arbitrary short time to let it "leak" down to zero (from the training example)
  RCTIME 2, 1, wStrength

  soundTime = wStrength * 3

  'DEBUG  "Raw strength: ", DEC wStrength ,CR
  '22wStrength = wStrength * wStrength

  wCurrentValueLocation = (wcount // wMaxValuesToStore) * 2
  WRITE wCurrentValueLocation, Word wStrength
  wStoredValuesCount = (wStoredValuesCount + 1) MAX wMaxValuesToStore

  wCount = wCount + 1

'  DEBUG "", DEC5 wStrength, "", CR

' DEBUG CLS, DEC5 sStrength, " soundTime: ", DEC5 soundTime
' FREQOUT 5,  soundTime MAX 250 , ABS (soundTime - 2640)

  RETURN

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'
Delay:
  PAUSE  wStrength MAX 250
  RETURN

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'
updateDisplay:
  IF index = 6 THEN index = 0
  LOOKUP index, [ %01000000,
    %10000000,
    %00000100,
    %00000010,
    %00000001,
    %00100000 ], OUTH

  index = index + 1
  RETURN

The world’s cheapest SCADA device?

As part of my learning about CygNet, SCADA, and the exciting world of Real World Data, I decided to build a SCADA-enabled device from scratch.

I wanted something that could be reconfigured to support any sort of sensor, was very small, and that didn’t require soldering.  A little bit of research (and by research I mean shopping) turned up the venerable BasicStamp: http://www.parallax.com/Default.aspx?tabid=295

It’s just about the simplest embedded system possible.  A processor with a built-in Basic (the language) interpreter, 16 I/O lines, a serial port, and a small breadboard for components.

For $80, you can pick up a version at Radio Shack that includes a complete training program.   (http://www.radioshack.com/product/index.jsp?productId=2117994&cp=&sr=1&origkw=parallax&kw=parallax&parentPage=search), or you can buy just the boards for anywhere from $10-$40 on eBay or the Parallax store.

I spent a little over a week going through the training and ended up with a photocell-based light meter that output readings on the serial port, and stored a small number of recent readings in its onboard EEPROM.  I hadn’t read a circuit diagram in about 30 years, so that was fun.

Useful tidbit:  the web has lots of resistor value calculators

ParallaxBasicStampSCADA002

Doug had already created a serial-device EIE for the Davis Vantage weather station, so it was a matter of hours for him to add a new “general” message that would allow communications with arbitrary devices

Protocol

The protocol is blindingly simple. Send a number (really a single byte with the ASCII code for a numeral) from 0-n, and the device sends back data and/or takes some action. The current command set:

  • 0 (48/0x30) – reset the device. Clear the ring buffer and all counters and restart the program
  • 1 (49/0x31) – return the most recent reading
  • 2 (50/0x32) – return all readings
  • 3 (51/0x33) – sleep (turn off power to the device for N seconds, where N is currently hardcoded)
  • 4 (52/0x34 – return a known diagnostic string (“deadbeef”). Why that word? It’s the default register & memory settings for the RS/6000 plus it’s all in hex. I didn’t say the commands made sense.
  • 254 – used in early testing

Example: Send “1” -> Receive “NNNNN” where N is a numeral

There is no way I could have gotten this running without help from everyone, but especially Doug, Pete, Sheri, and Derek.