Chapter 11: Meltdown
Content note: this chapter includes abusive/stalking behaviour (not within a relationship) and description of an autistic meltdown. It also contains consensual non-consent play.
It was a forty-minute taxi ride to Anjali's place, and she fell asleep almost as soon as she'd buckled her seatbelt. I couldn't blame her; she'd drunk much more than she was used to, and it had been quite a busy evening. I hoped she'd be able to enjoy a sleep-in next morning and not be too hungover. Meanwhile, I had a lot to process.
I had just offered the sexual services of my friend (and courtesan) to my other friend (and colleague), and she'd replied in a way I didn't fully understand. I suspected things between us at work were going to be all kinds of weird.
My friend (and courtesan) had just casually mentioned that she planned to leave the country at the end of her doctorate, about nine months away, and that had sparked an uncomfortable twinge in me.
Both those thoughts were taking up a lot of space in my brain, whirring endlessly like the pinwheel on that application that's never going to finish loading. But I needed to do my best to put them on hold, because the third thing couldn't wait. The very expensive facility management system that we'd built—that I'd built the guts of—was not working.
It wouldn't be an instant shutdown. The solver ran every half-hour, generating an updated schedule for the next twelve hours. They could work off that for a little while, but without the ability to run updates it couldn't adjust as flights changed or owners booked new shipments. It would gradually drift further from reality until it became useless.
There was a fallback system, of course. You don't put all your eggs in one basket with a project like this. After three hours or so they'd switch over to the fallback. But that was considerably less efficient, intended only to keep the most important activities going. It would require rejecting new contracts, which would get expensive quickly.
From what I remembered of the costs data, running on the fallback would cost something like fifty thousand dollars an hour, maybe more if we had to deal with cargo spoilage and missed deadlines.
If the cost of an outage is fifty thousand dollars an hour, and the taxi to Anjali's takes forty minutes, calculate the—
I did my best to squelch that line of thought. Unlike Mohammed and Jaden, I wasn't being paid to be on call. I had every right to do what I pleased with my Saturday nights, and I had a responsibility to get my friend home safely. That fifty-thousand-an-hour was Schiphol's problem and my employer's, not mine...or so I told myself.
It helped a little, but only a little. One's employer's problems have a way of becoming one's own, and besides—I'd built the damn thing. For it to be falling over, that was a personal affront, and I could no more ignore that than Anjali had been able to ignore being called a liar.
As soon as I'd given the driver directions, I called Lincoln. I told him what I knew—not very much—and that I was on my way in to the office, but I'd be a while yet. He listened until I was done talking, then asked, "What's your plan of attack?"
I'd been thinking about that in the shower. "Eyeball it quickly, see if there's anything obvious, but there probably won't be. There's a diagnostic that will try to identify what's causing it to fail, but that usually takes about twelve to fifteen hours to run, and the output is difficult to interpret. What I'd like to do is rent some cloud time and chunk it up into subset problems, run them in parallel on a couple of dozen instances, see if I can reproduce the issue on a simpler problem. Might cost a few thou but could save us some time."
"If you think that's the best option?"
"I do."
"Okay then. Is there anything I can do that's more useful than staying out of your way?"
Tell me I'm not an incompetent klutz and this isn't my fault and I'm not going to get fired.
"No... not yet. I'll keep you informed."
"Okay, thanks. I'll leave you to it. Call if you need anything at all, wake people up if you need to. Good luck!"
I hung up—realising too late that I'd forgotten to say good-bye, and hoping he wouldn't be offended by that, but there was nothing to be done about it right now—and tried to work out how to translate my ad-libbed plan into specifics.
Our container facility—I thought of it as ours, although really we were just contractors—was divided into five large wings and eight smaller ones, each served by between two and twelve robot cranes that ran on rails, offloading containers from incoming trains and semis, delivering them to their temporary home within the facility, shuffling them around as necessary to make way for other containers, and then loading them back onto the outgoing haulers when their time came. Cranes that size don't move quickly, and Schiphol is always busy.
The solver module was the software that figured out how best to coordinate all the pieces of that great big machine and decide what cranes and containers went where, when, so that everybody got their iPhones and Tommy Hilfiger on time, while ensuring that nothing collapsed or thawed. It was a beautiful machine, more intricate than the finest watch...and for some reason it had stopped working.
That was not just bad, but perplexing. I'd tested the system under a range of simulations, and it shouldn't just stop. Put too much demand on it, or take too many cranes out of action, and it would fall behind, with containers piling up. But we had automated checks to warn for excessive demand, and even if those had failed, it shouldn't just
stop
.
I tried to think of conditions that would make it fail altogether. If all the cranes in one district were out of action, that would do it, since then there was no way at all to load or unload there—but that would already have been detected by our checks. If there was cargo we'd committed to stack, and nowhere left to stack it without breaching safety limits—but again, there were supposed to be checks to warn well before we reached that kind of critical capacity. There were a few other cases I could think of, but nothing that seemed very likely. My biggest worry was that there
was
no real reason for the failure, that there was a bug somewhere in the fancy new algorithms I'd designed for this work. I'd tested them pretty thoroughly, but there were a lot of moving parts in there, and it was hard to be sure I hadn't missed anything.
Still, I phoned Mohammed and asked him to check the warnings log, just in case one of those checks had raised an alarm that had somehow been missed. Then I talked to Jaden about getting the cloud environment set up. We'd used something similar for testing, but not in parallel.
When I ended my call, the taxi driver coughed. "We're here, ma'am."
"Oh! Sorry! I was miles away." I had no idea how long he'd been stopped, waiting for me to finish up, and Anjali was still asleep. I shook her awake, and walked her to the door of her apartment complex. "You right from here?"
She fished out her keys. "I should be fine, thank you." She opened the door, then turned and gave me a hug. "Good night. Thanks for seeing me home."
"Sleep well." I wouldn't have said no to a good-night kiss, but she wasn't wearing her scarf and I wasn't sure quite what my two hundred dollars had entitled me to. Another time.
I watched until she was safely inside—the night had turned quite chilly—then hurried back to my taxi. It was another twenty minutes to my office, during which I did my best to stop trying to work out the root cause from first principles and instead focus on a methodical search strategy. It's not easy for me to think that way; I take pride in my ability to understand a system, and it feels halfway to defeat having to take a trial-and-error approach. But I was vaguely aware that this was how a sensible adult was supposed to handle this kind of situation, and while I might not actually
be
a sensible adult, I'd come this far by pretending to be one.
I will skip over the next few hours and spare you the technical details; they're probably not very interesting unless you're working for our competition, in which case I really shouldn't share them. Suffice it to say that we tested eleven different strategies that didn't work, and one that did—but because we were trying all of them in parallel, it only took us about four hours to figure out the cause of the bug.
It was a stupid, simple thing, as most bugs are. Wing H was one of the smaller wings in the storage park, dedicated to containers with special customs requirements. It didn't see a lot of business, and so it only needed two cranes. They had imaginatively been named Cranes H1 and H2. Unfortunately, due to a version control snafu, the solver module was under the impression that they were named H0 and H1.
The consequence of this was that the solver module didn't see Crane H2 because it wasn't looking for it. Unable to find the non-existent "Crane H0", it assumed it only had one crane to work with and did its best with that.
