《A Hardness of Minds》Chapter 7 Earth. Erosion

Advertisement

Dalton worked through the evening into the night. The possibility conquered his imagination. What if one could generate adversarial data so well, it would force the lander’s AI into making a poor decision—a fatal one? The only problem for him was the environment. The spacecraft was bathed in belts of Jovian radiation. Bad data, lost signals, it was all expected. One couldn’t just drop 3% of the data and expect the software to crash. The embedded algorithm had a concept of physics. Its mind was hardened with an expectation of bad data weighted against a consensus of correct data. It had millions of simulated orbits and landing. Dalton couldn’t fake fifty percent of data, the probe would still have a strong chance of success. Initially, the badness had to be subtle. The lying data needed to be correlated with reality but deviated at the tail. The fraudulent information smoothed down a slippery slope until a point of no return. To a point where no amount of correct data could save the lander, it had too small of a thruster to get back into a stable orbit. Once the small Ion thruster detached, it would land or crash. Bathed in bands of radiation from Jupiter, no parking orbit was safe for machine (or man).

He mixed positive examples with bad approaches. But there were still sharp discontinuities the AI might detect.

A gentile landing with 0.25 m/s vertical speed doesn’t just become a 100 m/s impact in a split second by setting the value to 100. The AI was too smart for that. They had trained it on millions of runs before leaving Earth. It would eject that bad data, reboot the radar, estimate vertical speed from its cameras or accelerometers or signal strength to the Europa Clipper. Anything and everything run instantly.

Nor could Dalton just invert the reward function: give it maximum reward for crashing instead of landing. It had a complex idea of ‘success’ saved in a large matrix that was not easily perturbed. Averages, convolutions, non-random noise. He had to confound it slowly over several generations. Tug the AI into a local minima where it thought it was secure. Until it felt the slam of cold ice onto the steel hull. Too late then.

Advertisement

But this was still purely theoretical to Dalton. Instead of using a generative adversarial network (or GAN), to develop the algorithm into a better one, he used a second hidden adversary to generate the bad data to feed into the good system to minimize the reward.

Dalton thought out loud (possibly to the sleeping cat, but mostly to himself). “Imagine: a forger makes fake money, and the Secret Service discovers the fake money. Then forger gets arrested, he doesn’t improve his con. The money’s good.”

The cat made no movement.“Now imagine Mr. Forger gets away without being caught. Eventually, the forger gets so good that the forgery is identical to the real thing. And caught only half the time, because the Service is purely guessing: Real or Fake.”

“What if the counterfeiter broke into the Secret Service’s office and swapped the real money, the trusted ground truth, with fake money?” Dalton put his hands on his head. “Well now, the agents would notice immediately. The initial counterfeit would look like a kid scribbling a paper dollar in green crayon. They’re trained.”

“But…” Dalton wagged his finger in the air. “Imagine the counterfeiter had someone on the inside who slowly slipped in remarkably good fakes into the training stack of money that the Secret Service compared to? Why, the counterfeiter could erode the sharp eyes of the detectives.”

So Dalton stood up a third algorithm whose overall maximal goal was the ‘long con.’ He let out a big hoot. The startled cat on the couch looked up. “Dalton H. Chatsworth, you are a genius.” He looked over at the two-tone cat, who had already laid its head back on the sofa.

He compiled the LanderAI Model on his machine and tested it on more realistic data. The first landing one was a failure (well, a success, depending how you looked at it). A few more runs, and tweaks of the hyper-parameters led him to a success. The simulated lander impacted at fifty meters a second and littered its digital pieces on a false Europa. The large horizontal speed spread them far.

Advertisement

“Ha Jim—and the Trillionaire—you’re only as good as my bad data!” He pointed his index finger at the screen. He was never serious about uploading it. Sure, he toyed with the idea. Even if he sent it, the AI wasn’t scheduled to be updated any more. A change freeze was in effect, but Dalton didn’t know the date exactly. But since they would make all the software public after the landing, he never was serious. Still, it could do something nasty, and this was already an amazingly difficult landing. Perhaps another minor tweak… and roll this change into the one CCS ordered me to make…

He opened up GitHub and saw his changes refresh but did not commit. There in the computer window, he irreverently mocked Jim, the one who has given him a break by selecting him as his undergraduate assistant. He also mocked the Trillionaire, who would certainly be remembered in the history books. More fame and likes unequally distributed to those who already had it.

Sure, Dalton worked hard and studied, and had earned his dues. But he was morally weak and information obese. Not a bad human, just a very average one. He might scorn the hand that fed him, but never would scar it. To top it all off, he was still something of a scientist and this was the top mission at the moment. Dalton had zero intention of committing it. This was hardly even a flirt with disaster! It was a car driving thirty miles per hour to a cliff a mile away.

He left his computer open and on the commit screen. He was thrilled and slightly success crazed. “Mr. Waffles, I’m going out tonight.”

Back in his bedroom, he got dressed. The only other thing he had that approached a hobby to others was gambling—well looking for exploits. He didn’t consider it gambling, if he could alter his luck.

In the living room with a suit and nice shoes, he spoke again to the disinterested cat. “I’m off to the casino to earn us some tuna.” Dalton slammed the door, and the neighbors heard him double-descending the stairs steps. Whump, whump, whump.

The cat looked around, got up and proceeded towards the useless electric box that commanded his owner’s attention.

He jumped up onto the table and looked at the keyboard. looks like a nice button to press, thought Mr. Waffles. Well, not really. Mr. Waffles was just a cat.

What the cat saw was a package of cat treats on the other side of the desk. Dalton did not fully shut the plastic zip enclosure, and the cat caught the scent of the treats. As Mr. Waffles walked over to knock over the bag, his paw hit the large button, committing the venomous code to review for all to see.

Dalton had always wanted to be known to the scientific community. The cat would make him infamous.

Europa, attempt no landings there.

    people are reading<A Hardness of Minds>
      Close message
      Advertisement
      You may like
      You can access <East Tale> through any of the following apps you have installed
      5800Coins for Signup,580 Coins daily.
      Update the hottest novels in time! Subscribe to push to read! Accurate recommendation from massive library!
      2 Then Click【Add To Home Screen】
      1Click