It’s been a crappy few months, frankly. One of the problems with “research” is that by normal standards, most projects going to be “failures” in the sense that you fail to achieve something. In the scientific and commercial sense, though, it’s not really a failure if and only if you learn something from your actions. This blog is here for precisely that reason, but for a couple months now I’ve been running into brick walls, several of my own construction, but at least we now know where a lot of walls are in the pitch-black Dungeon of IPv6.
Stephen H has been the other test monkey on the IPv6 infrastructure, and he’s gotten to experience vendors who don’t know what they’re talking about, vendors who ship the wrong things in the wrong order, and researchers (heh) who have eyes bigger than their stomachs, so to speak.
From the research point of view, the IPv6 problem is one coin with two sides:
- Is IPv6 practical, useful, and economically viable?
- Can we make our products IPv6 compatible in a cost-effective manner?
I’ve been been flipping from one question to the other as the frustration builds up to a point of intolerance. I couldn’t get IPv6 infrastructure working so I did coding. I couldn’t get coding to work so I did infrastructure. It’s a tunnel with two ends, both of them dark to date, but this week I’ve seen a we spark at one end, and I’m reasonably sure it’s not an oncoming train.
See my previous post on gumption traps, because that’s what the last three months have been for me, one after another.
The Fail Saga to date
- Two wrong ways to switch out the underlying CAddressBytes class – nothing stupid here, just experimentation, but since this particular style of work needs to result in running code, making a mistake, even if you learn from it, is expensive and annoying.
- A couple weeks of probably-not-so-sucky coding undone by a stupid VSS mistake (mine). Originally I’d been using the development VSS, and wanted read-only access, but there was some problem with setting it up that way so I ended up with a normal account, which means I could write to it. And I did. <mentally insert emoticon for stupid-face here> I have a completely separate VSS repository for Research projects, which I created a couple months ago when we started on the IPv6 stuff, and I’d replicated the entire dev VSS. No problem. Set up Visual Studio to talk to my research repository.
Here’s where the stupid (aka learning) part comes in. If you have multiple repositories, one of them is the default. At some point, somehow, I managed to re-set my default back to the development VSS. On top of that, VSS hasn’t been my native source control mechanism for a long time so I’m not used to saving projects back to it, and I did work on the SupportNet library without actually managing to put it in the repository. Eventually I noticed I hadn’t been saving it, and saved it to VSS. The wrong VSS. <bad word><worse word>. In a near-panic, I restored, I deleted, I reverted, I saved, and at some point, I lost the two files I’d been working on. <more bad words>. Fortunately I was able to revert my changes to the development VSS and then Derek did some magic to make it read-only for me, and my adventure was over.
- networks that mostly don’t – ongoing problems with VPN connections dropping in speed from 5Mbps to 3Mbps to 1.5Mbps (on a good day). Some of this was due to upgrades to Windows 7, because Cisco doesn’t support their *good* VPN client software, and the rest of it was just a freakin’ mystery of the ages.
Finally, Stephen and Will, sick of my shadow darkening their door, shoved a metal box in a cardboard box and sent it to me saying, “Plug this in and go away. Please.” They’re very nice, and I wish I didn’t have to spend so much time bugging them, and this box will do that. It’s an outer gateway/VPN router for my office network, which allows me to see the SLO offices transparently, without a VPN connection.
- IPv6 that doesn’t seem to work. Despite being pretty scrupulous about following various Microsoft documents, I haven’t been able to get IPv6 packets from one machine to another unless they were on the same physical network segment. Barfy. IPv6 makes bunny cry, some more. Being able to send packets locally is pretty meaningless, since no network in existence is actually configured that way. Very frustrating.
Fail saga fails, finally
…and then the sun came out. Admittedly, that meant it was –4 degrees F, but you can’t have everything. Where would you put it? In our case, we’d put it in the giantamous (it’s a word, really, I promise) IPv6 address space, and we did.
Note that the Appleton and SLO offices are connected over the public Internet. I signed up with a “tunnel broker”, a firm who helps provide “tunnels” for routing IPv6-over-IPv4 over the public Internet. At each end, a daemon pretends to be an IPv6 device and sends/receives IPv6 data by packing it into and unpacking it from IPv4 packets, which themselves are routed normally. Needful to say, this means everything takes more than twice as long, but connectivity beggars can hardly be choosers.
I have two tunnels: one between SLO<->Santa Barbara, and one between Appleton<->Chicago. The end points (on the Internet side) are public POPs provided gratis by various vendors/ISPs in an attempt to help foster IPv6 adoption.
Our tunnel broker is SixXS.net – After they approved the routing and rationale for my wanting these tunnels, I set up the virtual network devices (tunnel adapters) on frith’s external IP address and another on one of my development machines here in Appleton, taking care to route proto-41 (ports 41-44 and ports 58-60, actually, which are the IPv6-over-IPv4 canonical ports) to my dev machine. After a few minutes of hyper-ventilating, it worked. I could send IPv6 packets from Appleton to SLO and vice-versa, as confirmed by WireShark.
There were problems, though, and we need to keep them in mind. With the VPN router in place, I have an outer router/firewall (NetGear VPN FVS-338) and an inner router/firewall (DD-WRT on a WRT54G-TM). That combination didn’t work. Packets got dropped at the outer routers despite firewall rules saying “pass them to port NN”. This may have been because we’re having problems getting the outer router to see anything inside the inner router.
Other important things to note:
- both of the CygNet tunnel endpoints have fixed IP addresses. You can do this with dynamic addresses, but it’s unreliable, a pain, and did I mention it’s unreliable. Don’t do it.
- this wouldn’t be necessary if we could get IPv6 Internet service, but neither Time Warner Business Cable (Appleton), or <mumble> (SLO) offers it. In fact, they offer the Catch-22 version. “No one asks for it so we don’t sell it.” “We’re asking for it.” See statement #1.
It gets worse, in point of fact, because eventually, assuming ISPs switch to IPv6, you’ll need to run IPv4-over-IPv6 using similar tunneling mechanisms, although I suspect this will be a long time coming. If we’re lucky, we’ll see IPv6 in the next 3-5 years with major ISPs in major markets. ComCast announced just last summer (2009) that they’re going to be conducting tests of IPv6.
- Documentation about tunneling technologies is both abundant, incorrect, obtuse, and scarce. Bad combination. There are several tunneling technologies in use (IPv6-to-IPv4, ISATAP, and soon Ranger) on MS platforms. Docs are being written by different groups with differing levels of knowledge and authority. Very frustrating.
I remember being this frustrated when I was learning about IPv4, after having spent 10 years learning DECnet, both at a personal and a professional level. I know it will get better, but damn, I feel so stupid right now that it actually gives me headaches. I’m convinced that IPv6 is the way to go, and I wasn’t [at this time] last year, so I’m feeling less like the Sisyphean router, endlessly pushing infinite TTL packets into a local loop.
<diagram goes here – working on it after realizing I didn’t have Visio installed and debugged – Lynne>
Actually, this is a real Visio diagram using a stencil I got at http://www.visguy.com/, which also has lots of other useful Visio stuff (articles, stencils, tools). We (and by “we” I mean “I”) need (and by “need” I mean “want”) this for CygNet diagrams. Actually, you know, a CygNet Visio stencil would rule.
And Another Thing
Our IT guys? Inarguably the best generalist support team I’ve ever worked with anywhere, ever. I know people who know more about single subjects, and I know people who are nicer (well, I now people who bring me sandwiches, at least), but in all seriousness, I’ve yet to meet any two people with more patience and knowledge. Better, Stephen is getting a baptism-by-fire on IPv6 problems, which means he’s going to be valuable as a customer resource in all likelihood. Dudes, thanks.