# Engraver


Watch the video: [https://www.youtube.com/watch?v=kzIS3i_QYFI](https://www.youtube.com/watch?v=kzIS3i_QYFI)

---

> You can see pictures of a robot arm laser engraver attached.
Can you figure out what it is engraving?
>
> Note: the flag should be entered all in upper case. It contains underscores but does not contain dashes.
>
> Good luck!

## Overview

_Webster's dictionary defines engrave as "to impress deeply as if with a graver"_

As the challenge description states, we are provided with a picture of a robot arm holding a laser:


We even get a neato action shot showing it drawing the letter "G":


In additon, we are provided a pcap file.

### Context clues

Using _context clues_ we can infer that the information about which letter to draw is being sent from a [computer](https://en.wikipedia.org/wiki/Computer) to the robot arm over USB:


### Decoding

As mandated by CTF regulations, we first run `strings` on the input file and see if a flag pops out:

$ strings engraver.pcap
AMD Ryzen 9 3900X 12-Core Processor (with SSE4.2)
64-bit Windows 10 (21H2), build 22000
Dumpcap (Wireshark) 3.6.5 (v3.6.5-0-g21f79ddbefbd)
\\.\USBPcap2
USBPcap2
64-bit Windows 10 (21H2), build 22000
 jpC
 jpC
 jpC
 jpC
 jpC
 jpC
 jpC
 jpC
 jpC
 jpC
 jpC
 jpC

Hmm no dice... and unfortunately I don't understand the `pcap` file format...

Applying the Fast Fourier Transform to the raw file data reveals a low spike near ~429 Hz (B?, B/s??, BHz???) and several harmonics at multiples of that peak:


Unfortunately this is not at all useful and entirely the wrong step to take at this point in the solution.

Luckily [shark fren](https://www.wireshark.org/) can help us with `pcap` files!


### shark 🦈

With shark fren's help, we can see each individual packet being sent back and forth during the capture:


Neat! We can click on each row to see a different packet:


_Wow data!_

Most of the packets look something like this.

There's a USB header followed by some data shark fren doesn't understand and has remorsefully labeled "Leftover Capture Data."

Shoot, we're out of luck; we export the packets as json and switch to snake.

### snake 🐍

In Python, we can pull out all of the "Leftover Capture Data" fields (which are now called `usb.capdata` in the exported json) and print them:

import json
dat = json.load(open('./out.json'))

for pk in dat:
    try:
        u = pk['_source']['layers']['usb.capdata'].replace(':','')
        print(u)
    except:
        pass

We get:

5555080301f40101fc08000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
5555080301dc05021405000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
5555080301dc0503a406000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
5555080301dc0504c409000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
5555080301dc05054006000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
5555080301dc0506dc05000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
5555080301f401016009000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
5555080301e80302dc05000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
5555080301f40103dc05000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
5555080301e803021405000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
5555080301f40101fc08000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
5555080301dc05021405000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
5555080301dc0503a406000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
5555080301dc0504c409000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
...

Pattern recognition _activate_. This looks like:

5555080301[xxxxxxxxxx]000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

How about we just pull out the `x` part:

f40101fc08
dc05021405
dc0503a406
dc0504c409
dc05054006
dc0506dc05
f401016009
e80302dc05
f40103dc05
e803021405
f40101fc08
dc05021405
dc0503a406
dc0504c409
dc05054006
dc0506dc05
...

Ok so what do we even expect these packets to contain?

These are instructions being sent to a robot arm. So probably they contain information about how to adjust each servo.

So probably a position... relative or absolute?

Maybe some timing information... if you're moving two motors at once, you need to make sure they coordinate.

Each packet only contains 5 bytes so this is probably not enough information to specify all the motors at once... so there's probably a "motor index" somewhere in there.

We return to snake; let's dump out all of the byte options at each index in the packet data:

for i in range(5):
    print(f'idx {i}: {set(x[i] for x in data)}')

idx 0: {232, 220, 244, 144}
idx 1: {1, 3, 5}
idx 2: {1, 2, 3, 4, 5, 6}
idx 3: {64, 96, 226, 164, 196, 14, 220, 20, 120, 252}
idx 4: {4, 5, 6, 8, 9}

Interesting, interesting... index 2 seems to contain the numbers 1-6. That can't be a coincidence right. This arm also has 6 motors!

So we have `[??]m[??]`. Let's try decoding those other two parts as integers:

for pk in data:
    a = int.from_bytes(pk[:2], 'little')
    m = pk[2]
    b = int.from_bytes(pk[3:], 'little')
    print(a,m,b)

500 1 2300
1500 2 1300
1500 3 1700
1500 4 2500
1500 5 1600
1500 6 1500
500 1 2400
1000 2 1500
500 3 1500
1000 2 1300
500 1 2300
1500 2 1300
1500 3 1700
...

_Round numbers!_ We get a surge of dopamine. We must be on the right track...

But what do the numbers mean?

Let's view the packet data along with the actual time it was sent:

data = []
for pk in dat:
    try:
        t = float(pk['_source']['layers']['frame']['frame.time_epoch'])
        u = pk['_source']['layers']['usb.capdata'].replace(':','')
        u = bytes.fromhex(u)[5:10]
        data.append((t,u))
    except:
        pass

start = data[0][0]
for (t, pk) in data:
    a = int.from_bytes(pk[:2], 'little')
    m = pk[2]
    b = int.from_bytes(pk[3:], 'little')
    
    off = t - start
    print(f'[{off:.2f} s] {a} {m} {b}')

[0.00 s] 500 1 2300
[0.51 s] 1500 2 1300
[0.51 s] 1500 3 1700
[0.52 s] 1500 4 2500
[0.52 s] 1500 5 1600
[0.53 s] 1500 6 1500
[2.05 s] 500 1 2400
[2.55 s] 1000 2 1500
[4.37 s] 500 3 1500
[5.69 s] 1000 2 1300
[7.51 s] 500 1 2300
[8.01 s] 1500 2 1300
[8.01 s] 1500 3 1700
[8.01 s] 1500 4 2500
[8.02 s] 1500 5 1600
[8.02 s] 1500 6 1500

Hmmmm... some interesting patterns:

If we look at the first packet:

[0.00] 500 1 2300

The next packet doesn't come for 500 ms.

Then we have a burst of packets around the same time:

[0.51 s] 1500 2 1300
[0.51 s] 1500 3 1700
[0.52 s] 1500 4 2500
[0.52 s] 1500 5 1600
[0.53 s] 1500 6 1500

The next burst doesn't come until 2s (1500 ms later)...

Do you see where this is going?

The first number probably corresponds to motor movement time which means the third number is probably position related.

Let's plot the position values for each motor based on the time the packet was sent:


Wow motors 4, 5, and 6 are not moving at all! Motor 1 just alternates between two positions; and motor 2 and 3 are actually moving between a few positions.

## Simulation
Now that we have the packets decoded, we can tell what instructions the computer is sending. Unfortunately, we don't care! How do we convert these useless numbers into juicy shapes for our eyes to consume?


The problem is that these instructions have to be converted into movements by a robot that we don't have. To solve this, we attempted to set up (purchase from the internet) a system that could execute these instructions on bare metal. Luckily, [a quick search on Amazon](https://www.amazon.com/s?k=robot+arm) finds [an arm](https://www.amazon.com/dp/B074T6DPKX/) that, at least visually, seems similar to the provided image.


Unfortunately, though the arm itself was very fairly priced, nobody on the team could afford the same-day delivery necessary to transport this arm to us sufficiently quickly. Thus, we turned to the next best thing—simulation.

### Modeling the Arm

Because no pre-built tools for simulating such an arm existed, we had to build our own. The first step was to model the components of the arm so that the motion of the laser could be accurately computed. Thus, we opened up Fusion 360 and got ready to model.

But, we realized that, without proper dimensions, our model would produce distorted and inaccurate results! Therefore, we needed to search for technical schematics. To our dismay, we found only a few useful drawings. This image with rough dimensions was enough to get started, though:


Inside Fusion 360, we can create a sketch for the baseplate and apply the dimensions we found online as constraints. (Note: black lines are perfectly constrained, blue lines are not):


Next, we use the extrude tool to pull this surface into 3 dimensions:


Unfortunately, with no information about the thickness of the baseplate, we are left to guesstimate 5mm...

Modeling continues in this fashion. We construct sketches on planes, extrude profiles into 3D, construct sketches on those faces, extrude new profiles, etc...

For the LDX-218 servo, were were able to find some schematics online showing the mounting holes and overall profile:


We can apply these dimensions to our sketch:


Continuing, we design the U-bracket that mounts to the shaft of the LDX-218 servo:


In Fusion 360, we can construct a revolute joint between the motor shaft and body (and a rigid joint between the U-bracket and shaft):


Some coffee later and we have servos #4 and #5 attached to the central bearing (look at that inverse-kinematics solver go!):


To speedup the modeling process, we modeled the actual arm and the claw separately and then combined them later.

In order to ensure proper connectivity, we produced a technical document describing the motor-claw interface specifications:


The claw at the end of the arm was particularly challenging to model, as it had many small moving parts with no documented dimensions. However, since these parts serve no structural purpose, many of these dimensions could simply be approximated through careful examination of upscaled screenshots of the robot's mobile phone control interface.


Because of the angle of the provided photo, the servo used to actuate the claw was unknown. Since it seemed to come attached to the claw, the instruction manual did not provide a model number. Additionally, it would not make sense for this servo to be the same type as the others, since it only needs to exert a small amount of force.

Luckily, a [UIUC ECE report about a shoe sorting robot](https://courses.engr.illinois.edu/ece445/getfile.asp?id=14932) provided exact model numbers for each servo. These did not match our prior research, but the model number for the claw servo—LDX-335—allowed us to find schematics with reasonable dimensions. Once again, because the claw serves no structural purpose, we compromised and accepted these as correct. After adjusting the position of some parts of the claw by a few millimeters to account for this new information, we were able to move on.


Several hours later we arrive at a fully featured, fully rigged robotic arm:


Now it is time to throw all of that rigging data out and redo it in Blender!

### Blender Import

*Who knew moving files around could be so complicated?*

Now that we've got these models in Fusion 360, it is of course time to import them over into Blender where we can make them look :sparkles:pretty:sparkles:. How, you might ask? ~~STL export~~ nope; ~~FBX export~~ nope; ~~3MF export~~ nope; ~~STEP export~~ oh wait Blender doesn't read STEP and the addon costs *monies* - nope; you know what f\*\*k it we're doing STEP anyways

What could go wrong?

#### STL

Each STL file can only hold one object, so this resulted in a folder of 3716 files. Also, they don't have any texture information. So after import, we'd need to corral all 3716 objects into a sensible hierarchy *and* assign materials to all of them. Not happening.

#### FBX

This format supports assemblies, materials, and names for everything. But upon importing into Blender, there were way too many intermediate empties created for all the grouping relationships, so we went looking for better options. But it did many things right: we got the proper hierarchy of the assembly, the names of all the parts were imported, all the materials were created properly (in that all parts with the same material actually shared the same material), and the materials were even named.

#### 3MF

This one went alright, but required a 3rd party Blender addon. We had most of the features of FBX, with fewer empties, but also lost names on everything

#### STEP

STEP was a fun process... Blender didn't natively support it, and we weren't about to buy an addon to fix that, so this required a lot of fiddling.

It seems like pretty much everyone uses OpenCASCADE to deal with STEP format. Also, glTF is a (fairly new) format that is very featureful, supporting assemblies, PBR materials, and normals (it's designed for graphics). A quick Google search later points us to [a StackOverflow answer](https://stackoverflow.com/a/62872824) that mentions a curious "Draw Harness". More Googling tells us that the Draw Harness, also known as DRAWEXE, is a part of OpenCASCADE, used as a test harness and also as a (pretty old school) modeling tool. But we can use it to mesh the STEP into a glTF!


Results? The Y and Z axes were flipped (at the point of import of STEP into DRAWEXE), but that's easy to fix once we get it into Blender. But, we've got a sane hierarchy with a "minimal" number of empties (*only* about a hundred), properly linked materials, and names for all the parts. The only thing we were missing was material names; but we only had 6 or so materials total, so that was fine.


Not bad!

### Rigging

All in all, rigging was pretty straightforward: create an armature, position each bone so their tail is along the rotational axis of the servo it's representing, and then track down and reparent everything that's supposed to be attached to it.


The claw was a bit more fun. This was set up with an IK rig so that we could simulate the linkages when turning just the servo motor:


The bone for the servo is constrained to only rotate along its axis (although that's not strictly necessary - we're driving it with a driver anyways). The bone for the claw arm itself is directed from the end of the linkage bone, and is set to IK track a dummy bone created at the head of the servo bone. The other side has the same setup, except the "driver" bone is set to copy the rotation of the bone on the other side, so we get the nice claw action we want.

### Motor Keyframes

Once we had the motor data extracted with the help of *snek fren*, now we need to load it onto the arm itself.

*Snek fren* comes to our rescue again! With the power of the Python Blender API, this turned out to actually be quite a quick task. We can simply play through all of the motor commands in chronological order. For each command, first we keyframe the corresponding bone at its current position at the start time (which is holding the position set by the previous command). Then, we rotate the bone to the commanded position - since these are servos, we assumed the center position was 1500 (in microseconds, this is the standard center for PPM RC servos), and then eyeballed the range until the arm was level like our reference picture. Finally after having rotated the bone into position, we insert a keyframe at the end time of the command.


### Dynamic Paint

A challenge titled "engraver" without any actual engraving happening is no fun, so we decided to mod our laser pointer a bit to give it enough power to mark some anodized metal :)

Drawing the marked regions into the plate is the perfect job for Blender's dynamic paint system. It's very straightforward: add the plate as a canvas, the laser as a brush, and simulate! ...or so we thought.

It turns out this step took a good amount more fiddling with settings until we got good bake performance and output quality. The best brush turned out to be a simple flat square parented to the laser, set to project along its normal. (Apparently constraints don't work with dynamic paint subsampling.)

We used image data for the canvas instead of vertex data, since it was easier to adjust the resolution compared to subdividing the mesh to get enough vertices (and more efficient). Total bake time on 32 threads: about 20 minutes.

## Production

### Cool Visualizations

Blender is cool and all but once we render out a scene, we need to actually put it in the final edit. For that process, we turn to Adobe After Effects.

Now, when working with 3D data, we want to be able to add 3D effects, like things that stick to objects in the scene. Usually the solution here is to motion track your video in order to solve for the camera movement.

However, since our footage is actually a render, we already have a perfect camera solve and we can just import it from Blender!

We can use [this magical blender plugin](https://developer.blender.org/diffusion/BAC/browse/master/io_export_after_effects.py) to export our camera along with 3d data for some select object like our motors into a `jsx` script.

Inside After Effects, we just need to run this `jsx` script to generate a new composition with our 3d layers already configured. Then when we add our footage, everything lines up:


### Sound Effects

In order to truly understand our robotic arm we need to be able to hear it!

Luckily we already have the information about _when_ the arm moves, and we just need to play some servo sounds at each movement in order to generate some realistic sound effects.

Generating sounds at particular points in time is basically what MIDI is designed for, so let's just use that. We can use the Python `midiutil` library to generate our file:

from midiutil.MidiFile import MIDIFile

mf = MIDIFile(1)

mf.addTrackName(0, 0, "servo")
mf.addTempo(0, 0, 240)

for i in range(2):
    trk = notes[i+1]
    
    for off, pitch, dur in trk:
        mf.addNote(0, i, 60 + i, off + (dur * 0.5), dur * 3, 100)

# Initial large movement
mf.addNote(0, 0, 62, 1, 3, 100)

with open("servo.mid", 'wb') as outf:
    mf.writeFile(outf)

We can import this file into Ableton and start working with it:


In order to assign each note to a unique sound effect, we can assign a drum rack instrument to our track. For each note on the drum rack, we can sample from a partiular point in a larger audio file:


Then we just need to export the audio and bring it back into After Effects. As long as the timing is right, this effect is pretty convincing.

### Scripting

Graphs are cool, but you know what are cooler? _Animated graphs attached to 3d objects..._

So how can we do that?

In After Effects, basically every property of an object can be animated. You can do this either with keyframes (manual value setting) or with scripting. You can even import JSON data into your composition and reference it with a script.

In order to visualize where a motor will move next, it would be cool to have a few-second ahead line chart showing the next positions.

We can generate a path using a script like the following on the `path` attribute of a Path object:

var mdata = footage("export.json").sourceData
var dat = mdata[0];

// min and max for motor 0
var a = 2300;
var b = 2400;

// Controller layer has a few global params:
// - spacing: x-axis multiplier
// - height: y-axis multiplier
var spacing = thisComp.layer("Controller").effect("Spacing")("Slider")
var height = thisComp.layer("Controller").effect("Height")("Slider")

var points = [];
for (var i = 0; i < dat.length; ++i) {
	var x = dat[i][0];
	var y = dat[i][1];
	
	var n = ((((y - a) / (b - a)) * 1) - 0.5) * height;
	
	points.push([
		x * spacing,
		n
	])
}

// construct the actual path object
createPath(points, [], [], false)

Now this will generate the _whole_ graph. In order to see just the current slice, we can simply apply a mask and animate the location of the graph (sliding it underneath the mask):


Finally, both the mask and the graph are attached to the 3d object data we obtained from Blender which means they will move around the scene along with the motor.

# Conclusion

Overall fun challenge!

We were surprised how many solves this got seeing as how much work was required to accurately model and simulate a perfect 3D reconstruction of the robotic arm.

We are curious if Fusion360/Blender was the intended solution or if the author used a different CAD/Rendering combo.

Looking forward to the next GoogleCTF!

-- DiceGang 🎲