Revive the Underhanded Rust contest

[skade]

The Secure Code WG has expressed interest in doing another underhanded Rust contest.

The last one was not repeated due to personal constraints, not because it worked badly.

It had 2 issues:

• Lack of sent in examples. We ended up with 4.
  • A couple of people expressed that they felt not good enough in Rust to participate.
  • There was feedback that people had a hard time finding spots to hide things
• Lack of bandwidth of reviewers
  • Reviewers were community team members occupied with other things

The sent examples were all very interesting, though!

Announcing and promotion was not an issue, that's our day-by-day work anyways.

/cc @badboy @tarcieri

[Ixrec]

There was feedback that people had a hard time finding spots to hide things

This one is really interesting to me because it's sort of a victory: one purpose of the contest was to see how sneaky you could be in Rust, since it's supposed to be more secure by default than a language like C or C++. But then if we don't do contests because of it, we're not looking for new spots to hide things, which sorta misses that same point...

Maybe this means we should do it every N years instead of annually?

I also wonder if we should wait until the Rust 2018 changes have been fully absorbed by the community and the ecosystem before it's worth doing another contest, since we presumably want to see people abusing NLL and proc macros and default binding modes and so on to the fullest possible extent in the next contest.

[ckaran]

Could we automate portions of the contest? I'm thinking about the possible problems with proc macros, which all occur at compile time. The contest could have its own server, and the goal is to do 'bad' things to the server from within a package that is being compiled (kind of like core war, but against the server rather than a different simulated program). You could even have a small farm of virtual servers acting as a stand-alone network; in that case, some proc macros would attempt to break out of the compiling server and attack the other servers on the network.

Under this scenario, the contest could be run continuously, with different points being scored based on how much evil you're able to do, and against what versions of rust. E.g., evil against rust 1.0 is scored less than evil against the latest stable version. Not sure how evil against nightly or beta should be scored, as we can always remove bad ideas if they're caught in those channels, but once they hit stable, we're stuck with the bad ideas and have to find mitigation strategies quickly.

The only real issue I can see is getting all the dependencies of a package onto the server; if cargo generate-lockfile and cargo fetch can be trusted, they could be run 'outside' of the network to create a new virtual disk, then the disk loaded onto the servers. I'm hoping that this will ensure that the servers will be able to build the crate while remaining fully isolated from the internet, so that people can be assured that no matter how evil their code is, it won't escape the test network.

If it is run continuously, then we can gamify it, with leaderboards, etc. I know that isn't the goal of the competition, but the real challenge in anything like this is getting enough people interested, and keeping them interested. Otherwise the competition dies (along with any chance of getting outsiders to beat on rust to shake the bugs out).

[erickt]

I'd love to run another game. I was doing some research into a 2017 version, but I had a baby and it took up all my free time. Here's some of my notes from the time, if someone wants to draw inspiration from it:

Rust Community Contests

• Why didn’t many people participate in 2017?

  • Required too much in depth knowledge of the computer?
  • Rust is too safe
  • too hard

• Individual

  • Underhanded Rust style
  • Navigate a space as a virtual robot
  • Program a mario bot
  • Rust Plays Rust
  • Obfuscated Rust
    • worried we wouldn’t get any benefit out of making rust look bad
  • The Worst Rust contest
    • worried we wouldn’t get any benefit out of making rust look bad
  • uber the videogame
  • ghostbusters
    • If things have different values, then your first pass is get what's closest.
    • I'm not super excited about the potential Uber associations, but I like that sort of well defined but not immediately obvious how to solve sort of puzzle.
    • 10:01 AM Everyone loves 8-bit at the moment.
    • 10:01 AM In the UK, we have TV shows where people play 8-bit games (and new stuff, but meh).
    • 10:02 AM I can picture an Activision Ghostbusters / Sim City style map.
    • 10:05 AM But your second pass is some sort of distance-value metric.
    • 10:05 AM Your third pass is maybe, collecting multiple things on route.
    • 10:06 AM I seem to recall the Ghostbusters car could get full and you had to go back to the Fire Station.
    • 10:06 AM Oh man, this is going to ruin my evening.
    • 10:06 AM I have to run, I've got meetings and things.

• Competitive

  • Build it Break it
    • https://github.com/roryokane/build-it-break-it-fix-it-contest-fall-2014/blob/master/build/dist/spec/SPEC.md
  • Submarines hunting each other with sonar
    • https://boardgamegeek.com/boardgame/171131/captain-sonar
    • https://boardgamegeek.com/boardgame/142079/space-cadets-dice-duel

• Rust Golf

  • https://codegolf.stackexchange.com/

• smallest operating system

• Capture the flag game

• Collaborative

  • Rust Community Programs a robot
    • like twitch plays

• Alternative Reality Game

• stream processing something?

• program a space ship

Why in Rust?

• Could use any language. Could bound by either requiring wasm, or setting cpu/mem bounds to emphasize rust
• Requiring players to impl some Trait is a good introduction to the trait system and its advantages w.r.t C++ classes/inheritance.

Goals

• Can we incorporate webassembly?
• teaching
• bring new people into rust
• engage old timers
• I wrote 20 lines to impl some Player trait, and look, I can see my foo doing a bar.
• I think the point of a game like this is to help foster a sense of community, by giving people something to work on collaboratively or competitively

Other CTFs

• Stock Market
  • https://www.starfighters.io/
• http://theaigames.com/competitions
• https://www.riddles.io/
• http://aichallenge.org/
• https://www.meetup.com/preview/ACCU-Cambridge/events/240395637
• https://adventofcode.com/

Ranking

• https://rankade.com/ree#ranking-system-comparison

[skade]

@ckaran While I appreciate all of those ideas, I'd like to avoid a situation where we want too much, too soon :). What you propose would need some infra work involved etc.

Still, it might be a good kind of hobby project that could be adopted if it ends up being something. That allows it to be non-committed until the idea really works out.

[ckaran]

@skade You're right about wanting too much, too quickly. That said, I'd like us to have a vision for the future; I've written code before that had to be rewritten from scratch because I didn't think far enough ahead on how it could be used. Having a big pot of ideas + a plan on what will be implemented could help guide the initial design stages.