In the beginning...

The 4004

Like we did in the good ol' days

To someone used to today's MCUs, the 4004 will look mighty weird. To start with, it operates on 4-bit quantities only. The only flag is the carry flag. Instructions are mostly one byte long and take 8 clock cycles to execute. Some instructions are two bytes long and take 16 cycles to execute. Just to keep you on your toes, there does exist a single one-byte instruction that takes 16 cycles - FIN. However, that is only the beginning. The fact that this chip was developed for a calculator is quite evident in the fact that it has no logical operations at all. There is no AND, OR, or XOR operations at all. It is a one-operand instruction set, so the accumulator is usually the target of operations. So, if we have no logical ops, what do we have? ADD and SUB basically. To be precise, addition is always with carry and subtraction is always with borrow. This, again, shows off the 4004's "calculator" roots. For extra credit, and contrary to some of the docs you'll find out there, the usage of the carry flag during subtraction is quite weird... Some architectures treat the carry flag during subtraction as "borrow", as in: it'll be set if there was a borrow, and cleared otherwise. Other architectures treat the carry flag during subtraction as "not borrow", meaning that it'll be cleared if there was a borrow, and set if there was not. The key thing is that in every architecture I've evern encountered, it was one of those two options.

But with a twist...

The 4004 seemingly finds a third option. On the way in to the SUB instruction, the carry flag means "borrow", but after the SUB instruction, it means "not borrow". Yes, this is correct and I've verified this on the real hardware. I read one old newsgroup post (that I can no longer locate) where this was blamed on not having space for an extra XOR gate in the chip. The practical upshot of this is that to add a multi-nibble number to another multi-nibble number, simply clearing the carry up front and then using ADD on each nibble will work. To do a multi-nibble subtraction, one needs to not only clear the carry bit up front, but also invert it after each SUB instruction.

The 4004 is surprisingly register-heavy for such an early design. It has 16 internal registers, each 4 bits in size. PC is 12 bits long, and the hardware return stack is 4-deep. The current top element of the stack is used as PC so maximum actual function nesting possible is 3-deep. All together, that is 112 bits of state, which would be a whole lot of transistors in 1971, if you were to make that out of SRAM. It would take 672 transistors. That's quite a lot for a chip whose total transistor count is 2,300. So what did Intel do? They used DRAM for these bits! This is one of the reasons that the 4004 has a minimal clock speed, below which it will fail to work. This is rather unlike modern microcontrollers, most of which can operate all the way down to DC. Another slight weirdness is that 4004 has no interrupt support at all!

An unexpected luxury in a one-operand CPU is the presence of direct operations on memory operands. Well, only FROM memory, but still. There exists ADM which adds a nibble from RAM to the accumulator and SBM that subtracts a memory nibble from the accumulator. Other than those, all other operations only operate on the accumulator, sourcing data from the internal registers when a second operand is needed. Another somewhat weird thing is that while there is an instruction to load a register value into the accumulator (LD), there is not one to write the accumulator to a register. To do that one must use the XCH instruction that swaps the accumulator's value with that of a register. This is somewhat annoying since storing the same value into two registers now takes three operations instead of two. I guess that this was a compromise necessary to fit all the desired instructions into the encoding space provided by just-8-bit-long instructions.

So what else is there? Of course there is NOP, encoded officially as 0x00, but any byte less than 0x10 is treated as such. This is not a coincidence. The 4004 treats each instruction as being made of two 4-bit parts. The high part is called OPR and is sent on the bus first. The low byte is called OPA and is sent on the bus second. In general, OPR encodes the instruction and OPA is the parameter/index/etc for it. With this in mind, it is understandable why every instruction with the top nibble of zero is a NOP. Some instructions you'll find in the 4004 are pretty typical of early processors. IAC (increment accumulator) and DAC (decrement accumulator) make an appearance, of course, acting on the accumulator and setting the carry flag. But there is also INC which will increment a register and will not affect the carry flag. There is no DEC. Loading an immediate nibble-sized value into the accumulator is accomplished using LDM. Bit-shifts are always by one bit and always though the carry flag. RAR (rotate right) and RAL (rotate left) thus deliver precisely what they promise.

Since the carry flag is so often used, there are instrctions specifically for managing it. STC (set carry) will set it, CLC (clear carry) will clear it, and CMC (compliment carry) will toggle it. There is also CLB (clear both) which will clear carry as well as the accumulator and TCC (transfer and clear carry), which will set the accumulator to the value of the carry flag (0 or 1) and clear the carry flag itself. This turns out to be useful in all sorts of places, actually. Finally, there is TCS (transfer carry for subtraction) which is more useful for BCD math than it is for binary math. It will set the accumulator to "9 + carry" and clear the carry flag. I have found no use for this instruction yet in my code.

As far as weird instruction go, there are two more that are not all that useful. DAA (decimal adjust for addition) is one. If the accumulator is 9 or greater, or if the carry is set, 6 is added to the accumulator. Carry is set if the addition generates a carry, it is not affected in all other cases. This is also used for BCD math and is thus useless for my purposes. Another instruction of dubious value is KBP (keyboard process). It implements something like "count trailing zeroes", but only for powers of two. For an input of 0, it produces zero, for an input that is a power of two, it produces one more than the log base two of the value, for all other inputs it produces 15. This was meant to allow for easy keyboard decoding, I suppose.

There are two more ways to load immediates in the 4004, and both of them are a godsend for writing actual useful code. First, there is FIM (fetch immediate). This two-byte instruction will load 8 bits of immediate data into two consecutive registers, starting with an even-numbered one. The accumulator and the carry bit are unaffected, making this a nice way to load loop counter values into registers. A similar, in some ways, FIN (fetch indirect) will load two consecutive registers with an 8-bit immediate loaded from the code ROM's current page. As the 4004 is a Harvard-architecture CPU, the code and data spaces are entirely different and this is one of the ways to make constant tables work. Since using a single nibble as the address would not work, two registers are used, allowing for addressing up to 256 bytes of data. There would not be enough encoding space in the 8-bit instruction to encode a 3-bit destination register pair number, a 3-bit source register pair number, and the 4-bit OPR. Two possibilities existed here. One would be to use the same register pair for input and output. This would preserve the orthogonality of the instruction set but make actual use harder. The second (and what intel chose) was to hardcode one of the register sets. And indeed, FIN always uses r0:r1 as the address to read from, while the destination register pair is encoded in the instruction, and may, in fact, be r0:r1.

Control flow in the 4004

Memory and I/O in the 4004

The 4004 is not quite a complete processor. There are some instructions that it does not process at all. In fact, it does not at all process any memory instructions. Memory instructions are all whose OPR is 14. For them, the CPU will look at the top bit of OPA only. If it is set, the instruction is a read and during the X2 bus phase, the CPU will sample the bus and consider that the read value. If the top bit of OPA was clear, the CPU will place the written value onto the bus during the X2 phase. What is interesting here is that there are a few different read instructions and a few different write instructions, and the CPU knows nothing about how to perform them. When it sees an OPR of 14 (which will happen during the first bus phase fetching the instruction - the M1 phase), it will activate the CM-ROM and CM-RAM of the currently-enabled ROM and RAM banks, thus notifying all memory chips to watch the second nibble of the instruction (which will be sent during the next bus phase: M2). They watch the value of OPA (during bus phase M2), decide if they can execute this instruction, and if so, place the proper value or get the proper value from the bus during phase X2. So, one could argue that the 4001/4002/4289/4265/4308 memory chips are part of the CPU, since they decode and execute certain instructions. Intel used this to great success in the 4289 and 4265, which will decode many of the instructions in that space differently. One could even imagine a coprocessor that allows 4004 to execute custom instructions and transfer 4 bits of data per instr using this ability.

4001 and 4308

4002

The 4002 is the special memory for the MCS-04 system. It contains 320 bits of DRAM, refresh circuitry, and a 4-bit output-only port. I am not sure why intel made this decision, but this is so. Unlike the 4001, there is no mask ROM in here, so there would be no way to assign a "RAM number" to a chip. A different system is used. Each bank of RAM may have 4 4002s in it. So, logically, to determine a chip's index in a bank, we need two bits of information. One bit (the lower one) comes from the P0 input pin on each 4002 itself. The second bit is special per chip model. There are two: the 4002-1 and 4002-2. Thus a complete RAM bank, in order, will be made of: a 4002-1 with P0 = Vss, a 4002-1 with P0 = Vdd, a 4002-2 with P0 = Vss, and a 4002-2 with P0 = Vdd.

While the 4004 only has one CM-ROM output, it has 4 CM-RAM outputs, and as I mentioned above, this allows up to 4 banks of RAM without external circuitry and up to 8 with a single extra chip. So, a top-spec 4004 system without an extra decoder chip can have 16 4002s attached to it, for a total RAM capacity of 5120 bits (640 bytes). With a 3-to-8 decoder, the numbers double. In reality, memory addressing is not quite simple in 4004. So, if you were to only consider RAM you could address as a linear block that you could iterate over using a pointer, it is important to remember that each 4002 only has 256 bits of that. So a top-spec system would have 4096 bits (512 bytes) of that kind of directly-addressable RAM, assuming full 4 banks. I'll talk more about RAM addressing in the 4004 later. For now: each RAM bank is made of 256 nibbles addressable directly and 64 more, addressable weirdly.

So, what instructions does the 4002 handle? SRC, again, is used to select the chip in the current bank, both for I/O as well as for memory access. The top 2 bits of the 8-bit address sent select one of chips in the RAM bank. RDR, ADM, and SBM all read the nibble addressed by the last address sent via the SRC instruction. RD0, RD1, RD2, and RD3 read the status nibbles (more on them later). WR0, WR1, WR2, and WR3 write the status nibbles. WRM writes the SRC-addressed nibble - it is the opposite of RDM. The only other instruction the 4002 executes is WMP (write memory port), which sets the output value to be presented on the 4-bit output port of the currently-selected chip. The value will keep being outputted till another is written.

Sadly, there is no 4308 equivalent for RAM - there is no chip that would act like a full bank of RAM for the 4004. The 4265 can sort-of come close, but not in a fully compatible way.

4265

The 4265 is a general-purpose I/O device designed for the MCS-04 system. It has 4 ports of 4 I/O pins each and supports a number of modes of operation. You can peruse the intel docs on it at your own leisure to read about all the modes. I will only tell you about mode 12, as it relates to using the 4265 for RAM. In mode 12, the 4265 takes up an entire CM-RAM bank and responds to all 256 addresses that a SRC instruction might send. It can be interfaced to a 256x4 SRAM and it will read and write precisely like 4 4002s would. But wait, there is more, since 4265 in this mode also has 2 chip select pins that can be set to arbitrary values. If one were to use them as address lines too, one can interface a 1024x4 SRAM, which is a lot more RAM than a single RAM bank of 4002s could ever hold. Indeed, one would need to switch pages in this bank, as only one 256-nibble view is available at a time, but this is still pretty cool. The reason as to why this is not fully 4002-compatible is that there are no "status nibbles" here, so a 4265 + 256x4 SRAM is not a full replacement for a bank of 4002s if the code at all uses "status nibbles". When I talk about how they work and the advantages of using them, you'll see why this matters. Additionally, while the 4265 indeed has as many potentially-output port pins as 4 4002s would have, the way to control them is also not compatible (and, in fact, if you use 4265 for RAM access, you end up with no general-purpose output pins at all).

So, which instructions can the 4265 execute? SRC is decoded, as always, to handle inputting an address, of course. WMP (write memory port) is used to select the 4265 mode. In mode 12, RDM, ADM, and SBM will read the addressed nibble in the addressed page of memory. RD0, RD1, RD2, and RD3 select the given page, and then do a read. WR0, WR1, WR2, and WR3 select the given page and then do a write. WRM writes a nibble at the current address in the current page.

4008, 4009, and 4289

Given that intel will no longer manufacture you a 4001 with your custom contents (I called and asked), and the fact that the MCS-04 bus is rather strange and no other memory chip supports it, one might expect that one will need to do some perverted things to run code on a 4004 today. Of course, a simple FPGA, or even a modern microcontroller with a lot of level shifters could manage to pretend to be a 4001, but this is considered cheating in my book. Luckily, there is another way. There are two, even! Intel created a two-chip solution for a 4004 system to interface to normal garden-variety [[E]EP]ROMs: the 4008 and 4009. The 4008 handles the addressing part - it understands the MCS-04 bus protocol enough to tease out a 12-bit address that the CPU wants to read and can output that on 12 pins. Fancy that! The 4009 also understands the MCS-04 bus protocol, and it decodes memory instructions and generates control signals to handle I/O, ROM reading, and, optionally, writing in some weird ways that I prefer not to think about. It will also latch the 8 bits of data that represent an instruction and dish it out to the 4004 four at a time, as needed. The 4009 understands the same instructions as the 4001 does, except that instead of the I/O port being hardcoded at the intel factory, each write and read is output 4-wide to the outside world, to be dealt with as desired. This allows for a lot more flexibility. There is one more instruction that the 4009 understands that the 4001 does not: WPM (write program memory). This was meant for situations where the backing store was not [[E]EP]ROM but SRAM (eg for development). It works in weird ways that are beyond the scope of my lecture.

The 4008 and 4009 still need level shifters to connect to normal memories, since they were designed for the 15V EPROMS intel made, like the C1702A. The 4008 and 4009 are also rather hard to obtain nowadays. Luckily, intel also produced a combined chip - the 4289. It is basically a 4008 and a 4009 in one package, with level shifters built in. It can communicate with memories at 5V signal levels! This makes using a 4004 today pretty easy - a 4289 and a 5V 4096x8 [[E]EP]ROM is all it takes, really. The I/O story on the 4289 is also pretty simple and 5V-compatible. There is a pin that goes high when the CPU does an I/O read, and the 4-bit "I/O port" selection is available on 4 pins. Another pin goes high when the CPU does an I/O write, and the data appears on the I/O pins. In theory, this allows connecting up to 16 4-bit input ports and 16 4-bit output ports to the 4004, using simple buffers and decoders.

Using memory, and those "status nibbles"

The 4004 has no concept of addressing modes or even pointers, as I said. The way it addresses and uses memory is rather ... strange. Before you address memory, you need to select a memory bank. To select a RAM bank, one puts the bank number (0..7) into the accumulator and then executes a DCL (designate command line) instruction. This determines which CM-RAM line(s) go active during memory ops, and this selection remains active until another DCL instruction is executed. If no DCL is ever executed, bank 0 is used. There is no way to read back the current bank.

Each RAM bank (if fully populated), is made of 4 4002s. Each 4002 is made of 4 "registers". Each "register" is made of 16 addressable nibbles and 4 status nibbles. What do I mean by that? Well, if you were to read (using, say, the RDM instruction) memory in a bank from address 0x00 to address 0xFF, you'd first read nibbles 0 through 15 of "register" 0 in the 0th 4002, then the 1st "register" in it, then the 2nd, and then the 3rd. You'd then read the 0th "register" in the 1st 4002, and so on. Thus you'll have accessed every addressable nibble. Note that you did not access any of the "status nibbles" thusly. Those are accessed differently. With any address in a "register" selected, RD0 will read that 0th statis nibble attached to this register, WR0 will write it. RD1, RD2, RD3, WR1, WR2, and WR3 work similarly, as you'd imagine. Note that the status nibbles are thus not pointer addressable in the normal sense. They are also not accessible to ADM and SBM instructions, and thus to do any math on their contents you must first directly load them into the accumulator.

//these next two lines only needed if // the current bank is NOT already 3 LDM 3 //load 3 into accumulator DCL //select bank 3 //these next 2 lines are only needed if // the currently selected address is not // already 0xAB, they also clobber r0, r1 FIM r0, 0xAB //put 0xAB into r0:r1 SRC r0 //put r0:r1's on the bus as addr RDM //read nibble to accumulator

As I explained, accessing memory is a multi-step process. As the 4004 has no concept of memory addressing, that is left to whatever memory device is attached to the bus. All memory devices watch the bus to see if their CM-RAM (or CM-ROM) line is active during the X2 bus phase. If so, that means a SRC instruction is executing. All selected memory devices will receive 4 bits (high nibble of address), and then on the next bus phase (X3) they will receive the next 4 bits (low nibble of address). They will store this internally, and use this address for all future I/O instructions, until another address is sent using SRC. A curious little quirk of this is that every memory bank has its own "current" address, since only the selected bank will interpret a SRC instruction seen on the bus.

So by now it should be clear that to read a nibble at address 0xAB in bank 3, one might have to do something like what you see here on the right. Best case is just one instruction. This only happens if you already had the bank and the address selected. This is unlikely. Second-best case is 2 instructions. This only happens if you happened to have the address already in a register pair, so you only need to execute a SRC before your RDM. If you did not, you'll need to use a FIM and clobber a register pair to get the address into it before using SRC. And if you also are not sure you have the desired memory bank selected, you might also need to load the proper bank number into the accumulator and execute a DCL. Realistically, you'll often end up with the case of FIM + SRC + LDM, which takes up 4 bytes of code and 4 instruction times. Yes...slow

//add 32 bits at r2:r3 to 32 bits at r0:r1 //into 32 bits at r4:r5. Clobbers r6 CLC //carry cleared for first add LDM -8 //-loop iter count XCH r6 // ... into r6 loop: SRC r0 //set up addr for LHS INC r1 //increment LHS ptr RDM //get LHS nibble SRC r2 //set up addr for RHS INC r3 //increment RHS ptr ADM //add in RHS nibble SRC r4 //set up addr for DST INC r5 //increment DST ptr WRM //write DST ISZ r6, loop //loop

Now let's imagine doing some math (say an addition) on larger (say 32 bit) values. Obviously, we'll store them little-endian. This'll help us since math happens from LSB to MSB, and thus we'll want to increment the pointer. This is much easier than decrementing it, since INC instruction exists, but DEC does not. We'll also assume our value does not cross a 16-nibble boundary, which makes our address incrementation much simpler. You see that code here on the right. Quite verbose. The total useful work here is 1 carry clearing, 8 loads, 8 load-adds, and 8 stores - a total of 25 useful instruction cycles. This code will, in actuality take 91 instruction cycles. The main culprit, as is clear, is the need to constantly increment pointers and manually send them onto the bus. This situation gets a LOT worse if you are not able to guarantee that the values do not cross a 16-nibble boundary. In that case a simple INC will not do, and a more complex construction will be necessary.

The issue is not limited to math. A simple implementation of memory copying looks similar and wastes similar amounts of time selecting memory addresses and incrementing registers. As you can imagine, this gets slow very quickly. If you keep some global state in some variables, to access each you need to first waste 2 instruction cycles to load its address into a register pair using FIM, then use one more to send it onto the bus using SRC, and finally you may read it using RDM or write it using WRM. If you have a few global variables that are often used together and in a particular order, you could order them in memory such that accessing the second one does not require using 2 instruction cycles on a FIM, instead using a single INC on the lower nibble of the address you had already loaded. One instruction cycle saved, but you still do need a new SRC. Basically almost every memory access is a 2-instructions-minimum affair, due to the requirement of a SRC. For the nitpicky of you, yes, indeed, a read-modify-write of a nibble need not employ a second SRC, but this is rare.

This is where those status nibbles come in handy. Instead of storing your globals in normal memory, you can stash them in status nibbles. Then, a single SRC, targeting any of the 16 nibbles of the "register" they belong to enables them to be accessed directly using a single one-instruction-cycle instruction (RD0, RD1, RD2, RD3, WR0, WR1, WR2, or WR3). This is wonderful for data that is often accessed, and once you realize the speed advantages of these "status nibbles" you'll want to use them everywhere! This realization allowed me to speed up the 4004 MIPS emulator by a factor of 30%!

Performance and clocking

Some more annoying weirdness

All MCS-04 components operate at a very strange voltage level: their supply voltage is minus 15 volts. Yes. They also use inverted logic on all pins. To indicate a zero, a pin will be grounded, to indicate a one, a pin will output negative 15V. So to any other chip, even with level shifting, the signals will all appear inverted. Chips do not really care what you call "ground", so instead of thinking that MCS-04 chips need "-15V", it is simpler to think of them needing "-10V" and "+5V" supplies, and they are just missing ground pins. This helps in systems that also contain the 4289, since with this exact setup it can interface to normal 5V [[E]EP]ROMs. This just leaves you with the somewhat-annoying problem of generating a few watts of -10V...

Initial planning

Let's make a dev board

Emulating my 4004 system

The MIPS emulator

Why MIPS

A start

I started with emulating just the CPU, to evaluate how much space that would take and help me estimate the feasibility of the project in general. As this was my first time programming in 4004 assembly, I had no feel for how dense the code would be. Initially, I skipped dealing with RAM and just assumed that the "current" MIPS instruction will be in r8:r9:r10:r11:r12:r13:r14:r15 registers, MSB-to-LSB. Yup...half of the registers are used just to hold the instruction. I considered using memory for this, but the values would need to be used in many ways in many many places during decode, so that would turn out to be messier. Plus, I still had 8 registers left, that is two more than x86 ever had! Of course, "assume the instruction ends up in registers" is not testable, but that was not yet the goal. Initial dispatch (based on the top 6 bits of instruction) to a 64-entry (128-byte) table of unconditional jumps took 13 instructions, including the JIN that it ends with. So 128 + 13 bytes just for that. That is already 3.4% of the entire code space I had. Not a great start. Top level opcodes 0 and 1 each need another sub-table to decode. One table will have 32 entries and the other will have 64. They will need 12 and 14 instructions respectively to calculate the jump target. Thus once we've more or less handled the majority of the decode, we've used 128 + 128 + 64 + 13 + 12 + 14 = 359 bytes of code space. That is over 1/12 of the entire code space, and we have not yet even executed anything. Yeah... It was, of course, approximately at this point that I realized that this project will be harder than I had anticipated. But, no surrender!

Logical ops

I have never before seen a CPU that lacked ability to do basic logical operations, until I saw the 4004 manual. The 4004 lacks ability to do any of them. There is no logical AND, no logical OR, and no XOR. Intel, helpfully, gives sample code in their 4004 programming manual to implement those logical ops on nibbles, but I was able to produce more compact and faster routines. Nonetheless, it takes dozens of cycles PER NIBBLE to do this! How does one even do this? Observe that if we were to isolate a bit from each operand into a register's lower bit (the higher ones being zero), and then add those registers while input carry flag is zero, the result's bit 1 would only be high is the input's two bits were both ones (AND). The result's bit 0 would only be high if the input's bits differed (XOR). If we did the addition with input carry being one, the result's bit 1 would be high if either of the inputs was a one (OR). This is the basic building block of implementing logical ops in the 4004. The rest is looping and shifting! And then, you remember that each MIPS register is 32 bits long, and a whole lot of cycles are going to go into doing all of this per-bit!

In addition to the usual suspects of AND, OR, and XOR, MIPS also has NOR. Luckily it is easy to compute in a similar way. One might ask if there is a way to speed this up using some sort of a lookup table? Yes, but a table with 256 entries is 1/16 of the available code space. Three such tables is 3/16. That is a lot of code space to give away in a project where I was not sure I could even make the code fit in as is. So this idea was shelved.

Shifts

MIPS has the usual complement of shifts: left, arithmetic right, and logical right. They can be by a fixed amount (encoded in instruction) or a variable amount (taken from a register). The second part of that is trivial, we can find the right register and read its value. To the emulator, all shifts are by a variable amount between 0 and 31. Again, the 4004 makes this rather hard. The only shifts it has are shifts by one through the carry flag. So to shift a 32 bit value by one bit, we need a loop with 8 iterations. Thus to shift by N bits, we'll need to run that loop N times. This is getting pretty slow, eh? But wait, there is more. Arithmetic shift right requires the new MSB be the same as the last MSB. The 4004 lacks a way to do this easily, so it takes a few extra instructions to set this up every iteration. Thus, shifts are slow, and they get slower as you need to shift by more bits. One could come up with many clever ways to optimize this, but I was optimizing for code size above all else!

Space optimization and 4004-specifics

As you may recall, I mentioned that the 4004 has 4 levels of stack, and that one of them is always used as the current PC. This means that if you call into 4 levels of subroutines, you'll not be able to return all the way out to the last. This is annoying, but palatable. However, this also has another fun consequence. Because the 4004 treats the stack as a 4-entry circular buffer, you CAN have an unbalanced number of calls and returns. I use this to save some space in my code. When a MIPS instruction's destination register is $zero, the result is discarded. For speed and code size, I emulate an actual $zero register, and just ignore writes to it. This is faster and simpler than replacing all reads with zeroes as on MIPS more registers are read than written. Now, you might imagine having a isZeroReg() function, and after it, a conditional jump based on its output. If it says "yes", go handle next instruction; if "no", continue processing the current one. This would be suboptimal, since every callsite would need this conditional jump. My idea is better. My checkZeroReg() just goes to the handle_next_instr label if the destination register is $zero, and does not actually return at all. It only returns back to the caller if the desitnation register is not $zero. This means that every instruction targeting $zero pushes yet another value unto the 4004 stack that will never be popped. However, as long as you do not have too many returns, it is safe to have too many calls. This saves three bytes at every callsite, which there are close to a hundred of. Saves three cycles too! This was a big deal in my increasingly-cramped ROM.

There is curious little part about emulating MIPS: When, exactly, can you stop working on an instruction that targets the $zero register? For instructions with no side-effects, you can skip doing any work at all. So for example an ADDIU or an SLL can be skipped entirely. This is not true for instruction that might trap, like ADDU. Here, some work is needed up front - to check if the instruction might overflow and thus need to cause an exception. In my emulator I do this check, and then skip the actual addition if the target is $zero. Why someone might have such an instruction in their binary is another question, but as an emulator writer, correctness is important. Memory load instructions are similar. They might cause an exception, so even if they target $zero, the memory translation and access need to be executed to make sure they succeed (or to cause them to fail as they should). The only part that can be skipped is the final copying of the loaded value to the destination register, and the potential sign-extension.

Hypercalls

Second-order hypercalls

But, as I had mentioned, I developed much of this code not on real hardware but on an 4004 emulator I wrote (u4004). This made development easier, since I had not yet even built a board with a real 4004 yet. Indeed it is emulators inside emulators. It is emulators all the way down, in fact. In any case, the 4004 emulator also had hypercalls. Initially, before I properly emulated the SPI-attached SD card, UART chip, PSRAM, and VFD, they literally accessed the host file that pretended to be the SD card and printed to console to display text via hypercalls. This allowed me to focus on the actual emulation bits without worrying about the accessing the real world.

It is tight

SD card driver and the last of the ROM space

So, I added a virtual SD card to u4004, connected to a virtual SPI bus, the three output pins being three pins on a virtual 4002, one input pin being the lowest input on a 4289. I then went on to write what I believe to be the world's smallest SD card driver in existence. It fit into 190 bytes and would successfully init a card, get its size, and allow sector read and write. I also tried this driver on my dev board from earlier, connected to a real SD card, and found that it worked! Woo hoo!

I had 10 bytes left in my ROM and a lot more code to write. I scrounged hard and made a little more space - 44 bytes of ROM were free. I then re-read all the docs and saw that on reset, every 4002 will zero its memory, so I did not need to spend code clearing memory on boot. This saved another 12 bytes. 56 bytes free! Still, this was clearly not enough for what I had left. Failure!

The emulator needs more ROM

How to make ROM banking work

Well, OK. I can have 8192 bytes of ROM, in two banks, flippable by a pin controlled by a 4002. Jumping between them would take a little work, but it could be done. And there would finally be more space! The way the bank switching would work is that after the bank output pin is written using WMP, the next instruction would be fetched from the other bank. This means that the call-gates had to be precisely positioned in both banks. Additionally, 4002 outputs reset to outputting 0, which in MCS-04 means the higher output voltage. Practically, this means that the board would boot from bank 1 and not bank 0. Oh well, that is solvable with a cross-bank jump. Calls between pages are also a bit complicated, since the return value is only valid in the page where it originated. So instead of a JMS to a function, now I'd JMS to a veneer that would switch banks, continue in the other bank, JUN to the function, it would JUN at the end to a return veneer, which would swap the banks back, and only then BBL to return. This mess was necessary to not burn one of only three call depth levels available in the 4004. Messy but it worked!

Now that I have space...

Suddenly I had mode space! So many possibilities opened up! So many pieces of code that had once been optimized for space could now be optimized for speed. There were limits, of course, since jumps between ROM banks were a pain, so only a few things initially got moved. The first were the logical operations. AND, OR, and XOR each got a full 256-entry lookup table in the second ROM bank. How does one implement a lookup table on the 4004? Since the result is a nibble, just a table of BBL instructions is good enough, at a 256-byte boundary. A jump into that ROM page at an index whose high nibble is one input nibble and low nibble is the other input nibble would jump to the BBL that would populate the accumulator with the result. However, if the entire ROM page is filled with BBL instructions, how does one jump to them? Recall that the computed jump instruction JIN had a curious footnote in the manual: if it is the very last instruction in a ROM page, the address that the jump is relative to is not the start of its page but of the next. The 4004 manual warns that this is hard to use and should be avoided! Well, I found it wonderfully easy to use and used it to great success. Then, the functions to AND, OR, or XOR full 32-bit values were simple to implement and ran much faster.

Multiplication was another case where a table could help, but this is slightly more complicated. A nibble times a nibble can produce up to a byte of result (0x0f x 0x0f = 0xe1). This means that the trick with a JIN at the end of a page followed by a page full of BBLs with correct value would not work. Well, there is also the FIN instruction which loads a whole byte from ROM into a register pair. In fact, it also has the same quirk with regards to the addressing, so that placing it at the end of a ROM page would indeed allow using the entire next ROM page as data. Wonderful! There is one problem: FIN is not a return statement, so after it performs the load, it will continue executing the next page's data as instructions. This is most unpleasant! There is no way around this in the general case, but I am not seeking to solve the general case. If we imagine a LUT for multiplication, the first 16 entries will be zeroes. So, if we can simply assume that our larger multiplication implementation does not call the LUT for zeroes, then we can simply FIN and then BBL safely. This is what I did, in fact. Multiplication got 8x faster compared to the one-bit-at-a-time implementation I had had before.

Hardware

At this point, it was becoming clear that the project was feasible, and so it was time to build some real hardware! I decided that the final result should be artistic, recall the 1970s, and be able to be hung on a wall and look pretty! The board would be composed of all through-hole components, thick right-angle-only traces and no vias anywhere for a classic look. Moving on to parts selection...

SPI PSRAM

The VFD

The UART

I was not going to implement a full keyboard out of buttons, though. I thought that this would be unsightly, plus I am lazy! I did, however, plan on having a real serial port on the board. There was one problem: despite much searching I found only one through-hole SPI UART chip (the MAX3100), and, sadly it had a fatal flaw. While it supported doing flow-control signaling, it lacked the ability to automatically signal the other side to stop talking when local buffer was full. Instead, its flow control outputs functioned like GPIOs that the host had to control. This was a nonstarter for me, since my host CPU was too slow to do this fast enough. After much soul-searching, I decided that the UART will be the third surface-mount component on this board. Given the option to use an SMT part, I decided on SC16IS741A. It has a large 64-byte buffer and can do flow control automatically, with threshold settings on when to signal stop and when to signal resume. Awesome! I do not use flow control on the TX side since my emulated MIPS CPU simply cannot produce data very fast.

The blinkenlights

Now that I had curses, I could add more things to u4004. As I have mentioned, the emulator was bound to be rather slow. So, why not, for extra retro flair, show the current PC using 32 LEDs? And if I were to do this, why not also emulate it so that I could verify correctness? I did! For speed, the emulator updates this only every 32 emulated MIPS instruction, which is plenty.

The easy level shifting

I planned to use the TEST pin on the 4004 to indicate that there was a character ready to receive from the UART chip. This would allow easily checking for it, using JCN T conditional jump. This level shift from 5V to 15V was done using a FET and a resistor - simple stuff.

To convert my various 3.3V signals to the 5V that the 4289 will accept as inputs, I used a CD40109B. It has a high input resistance and a low output resistance allowing me to play various tricks with resistors on the output and not worry about anything.

Actual RS232 serial port signaling requires some rather high voltages. There are standard chips for dealing with that, and I chose a cheap through-hole one: HIN232. It just needs a few capacitors to work. It converts two lines in each direction between high voltage inverted and low voltage non-inverted signaling. This allows for transmit, receive, and flow control in both directions to be level-shifted.

The hard level-shifting problem

Power supplies

The garbled text mystery

More MIPS emulator fun

Memory translation

MIPS R3000 has 64 TLB entries, but Linux never does anything that requires this exact TLB entry count. This makes sense since it only uses indices it gets from TLBP (tlb probe) and indices pre-populated by the CPU itself on exception. When it writes a new entry, it uses TLBWR, which writes a random entry. This indicates that Linux might be able to cope with having fewer TLB entries. I tested this, and it worked precisely as expected. I decided to go with 16 entries, since this makes it easy to address each entry with a nibble. This also means that one full RAM bank would store all the TLB state. Cool.

Debugging the emulator

Predictably, a brand new emulator, written in assembly for a new platform will have bugs. u4004 made debugging easier, since at least I did not need to use real slow hardware. Additionally, it could be instrumented to understand the deeper emulated MIPS system. I commandeered a few unused 4004 opcodes to mark a few important places in the emulator. One of them was the place where a new MIPS instruction had just been fetched. This opcode will be ignored by a real 4004 (treated as a NOP). u4004, however, knows the memory layout of the emulator and can do more checks. It has the option to log the emulated MIPS state to console for easier debugging. It can (and does) also check the TLB state for consistency. Getting this right took a lot of work, so having this auto-checker was worth its weight in gold. If it notices an inconsistency, it will abort the emulation.

What good is aborted emulation? Well, u4004 also records the state of the entire 4004 CPU every single cycle. On abort, it will print out the state for the last 16,384 4004 instructions, allowing a lot of backtracking to see what went wrong to cause the inconsistent TLB state, why, and how. This proved to be instrumental in a few tough-to-catch bugs my MIPS emulator had. Sixteen thousand 4004 instructions is, usually, at least 10 MIPS instructions. As the TLB checker is engaged after each one this allows for a lot more lookback than should ever be needed.

MOV

MIPS has no MOV instruction. To copy a register to another one, a number of instructions can be used: an addition of zero (immediate or register), a logical or exclusive OR with a zero (immediate or register), a left shift by zero bits, a logical AND or OR of a register with itself. Different compilers do different things, and I tried to make sure my code had fast paths for the most common ones: shift left by zero bits and immediate add of zero.

Playing tetris

As conditional jumps have limited range that is also not symmetric (depending on the position of the jump instruction), code placement is crucial. This is also true for jumptables which must start at the start of a ROM page. It is equally crucial for LUTs that also need that same alignment. This all means that code gets moved around a lot, and every time some piece of code changes size, other pieces need to move around to maintain reachability of jumps. I had to do this a lot during development. You'll see a lot of ORG directives in the assembly that align code to proper boundaries. Sadly, the A04 assembler does very little checking around this, so an ORG directive that moves the current output address backwards does not get treated as an error, instead blindly overwriting the old bytes at the same output address. This caught me by surprise a few times during development, causing crashes. I advise you to be careful if you try to modify the emulator.

Speed optimization

Methodology

Now that I had a working emulator of my actual board I would build, I could run multiple versions of firmware on it to evaluate speed it would have had on real hardware. This was much better than testing on the real board, waiting for days for a single boot to complete (or fail). I did a lot of this, trying many things, incorporating improvements into the firmware, tracking boot time to first shell prompt. Once the emulator emulated a real SD card and real SPI PSRAM without hypercalls, the expected boot time could be measured and would have been around 8.9 days of real time on a real 4004 at 740KHz. My initial goal was to get the boot time under a week. Looking back at my log, some of the major wins on this were: lookup tables for logical ops and a lookup table for multiplication. This brought the expected boot time down to 8.4 days. At this point in time, I added some profiling to the emulator and had it print the hottest instructions. This provided some guidance!

Fetch

The longest part of emulating any instruction turned out to be reading it from memory. I considered adding an i-cache but this would require keeping the PSRAM chips' select line active even longer. Since I was already out of spec by a factor of a thousand, I did not wish to push my luck. It would have also required more RAM, and this was counter to my desire to minimize RAM usage to make the project more affordable. So what could I do? First, I unrolled the loop that received a nibble of data from SPI PSRAM, then I did the same for the code that sent a nibble. This lowered the boot time all the way down to 7.25 days!

Memory copy

Since it is impossible to pass 32-bit values in registers in 4004 assembly (that would take up half the entire register file), data is often passed to functions in specified memory locations. This means that quite a lot of time is spent copying data into and out of those locations. From the very start I made sure that no data crossed the 16-nibble address boundaries so that a simple increment could be used to access all variables, but even then, copying 8 nibbles took a while, especially between memory banks. Now that I had more ROM space, compared to when I initially started intending to fit into 4KB, I specialized some copies, unrolled some as well. The boot time dropped to 6.63 days!

More RAM

I noted that I had one pin free on the output port from the 4002 that drove my PSRAM. I wondered if I could add another PSRAM, to give Linux more RAM. Adding it was not too much work, mainly having to do with decoding the address to sort out which PSRAM to activate. Since Linux will handle non-contiguous physical RAM space, I could take advantage of this to make decoding easier. So, my first RAM chip is emulated at 0x80000000, while the second starts at 0x82000000. This particular arrangement allows the assembly code to quickly go from address to proper port value to select the correct chip. I also added code to probe RAM size in my MIPS bootloader (more on which later) and pass this to Linux.

Sadly, adding more RAM slowed the boot down. Linux creates data structures at boot that track physical pages and with more pages, more of them had to be created. Some kernel data structures are also dynamically sized based on available memory, and that also suffered. At this point in time with 16MB of RAM, boot time was projected to be 7.19 days. Back over a week! Womp!

Clawing it back

I was not going to give up so easily. More specialized memory copying routines were created, instruction dispatch got better by a cycle here and there. It was getting close - 7.03 days. I had an epiphany then. Why have a separate memory storage for the current instruction if I was already going to load it into registers? If I carefully track their liveness, I can skip this entirely and just load it into the same temporary storage that all memory loads go to. Down to 6.50 days!

Better shifts

Recall that I initially implemented logical shifts as one-bit-at-a-time, as a sacrifice to my lack of ROM code space. Well, now I had more space, so it was time to improve. I rewrote the shifts to first do full-nibble copies for any part of the shift that was a multiple of 4 bits, and then do bit-by-bit shifts for at most three iterations. For left shift this was trivial, but for arithmetic right shift this required some careful consideration for proper sign extension. After some fun debugging of some cases where I got it wrong, it worked. Boot time was projected at 6.19 days!

More RAM access unrolling

There were a lot of things I could do now that I had more ROM space. Unrolling the PSRAM address-sending code, which ran on every access had some nice pay-offs. The loop control was not very much there, but for code that runs a lot, every cycle matters. How much? Unrolling that loop with 6 iterations lowered the boot time to 6.01 days! That is not bad at all for some copy-n-paste work!

A look at WHAT runs

The Linux kernel contain{s,ed} a whole lot of nonsense that this system would never need. Nobody is going to be doing TCP/IP at this speed, nor does it really need support for esoteric filesystems or antique syscalls that nobody has used since the 1990s. I went to town removing kernel configs that were of no use! The kernel size shrank down to about 2.5MB and the boot time dropped too. A part of the drop was simply the time it took to load the kernel to memory, but another part was the kernel no longer initializing subsystems that would never be of use. The kernel also creates a dummy console in RAM to log to, even if told to log to serial console. Managing that and virtually "scrolling" it took noticeable time, so I configured it to be 1x1. This made a noticeable boot time difference too!

Almost all of my initial testing was with init=/bin/sh kernel command line, but this is not fair, since this does not leave you in a good state, with no session, no $PATH, no /proc or /sys, etc. On the other hand, using a real init would take months, since the password hashing itself would take that long. I wrote a tiny init (init=/sbin/uMIPSinit) that would set up a sane session, mount /proc and /sys, set the hostname and $PATH, and finally launch sh repeatedly as it dies. The sources to it are in the disk image at /root/init.c. Obviously this made the boot slower, but the smaller kernel made up for it. Boot time was 5.33 days now!

Linux supports block devices over 2TB in size. I had a hunch that disabling this support would help with speed. Why? A 2TB device of 512-byte sectors has sector numbers not representable by 32-bit numbers, necessitating 64-bit math. 64-bit math on a 32-bit MIPS CPU takes a lot of work, so I had a hunch that avoiding it would help. It did, but it also presented a fun problem. My rootfs, being a garden-variety ext4 filesystem had (as is default) huge_files feature enabled. This had to be disabled to allow it to be mounted read-write on a kernel with huge block device support removed. It was all worth it though! The boot time dropped to 4.81 days!

Fetch again

As a special last-ditch optimization, I added a code path for instruction fetch from SPI PSRAM, this code path assumes the read is 32 bits in size and that it targets SPI PSRAM. This is sane because the virtual system has no other places to run code from. Avoiding checks for need to sign-extend and for size loops added a small additional speed benefit: 4.76 days to boot! This calculates out to being around a 70Hz MIPS machine if the 4004 is run at 740KHz.

On host CPU speed

As mentioned before, each 4004 instruction takes either 8 or 16 clock cycles to execute. The 4004 is specified to run at 740KHz. Intel wanted to ship it at 1MHz, but apparently it couldn't perform at that speed across its entire temperature and voltage range. On my board, however, I am overclocking it to 790KHz with no issues. This is accomplished by running the 4201 in "divide by 7" mode with a 5.5296MHz crystal. For a Linux boot with my emulator, the actual 4004 instruction mix is 8.8% 16-cycle instructions, 91.2% 8-cycle instructions. This means that on my Linux/4004 board the effective speed of the 4004 is 90,640 instructions per second! I am not yet sure if this is a cause for celebration or tears.

Hardware cost optimization

"Affordability"

Throughout this project, ability for someone to replicate my work was my top concern. This is why I avoided using a 4265, for example. I could have avoided using the 4201, but the alternative methods of generating the clock were quite complicated and not very precise clock-speed-wise. Intel's recommended clocking schematic is shown in a previous section above. It is quite a mess. I simply did not want to do that, so I chose to use a 4201. Using the 4289 was a simpler decision. It is much easier available than the 4008 + 4009 combo. I considered designing the board to accept either of those, but lacking a 4008 to test wish, I decided to not risk it. I did design the board to accept a 4040 instead of a 4004, and verified that this works. No extra capabilities of the 4040 are used, to maintain the 4004 compatibility. In reality, the benefits would not be great, anyways.

The prices for 1971 chips

Sadly, you'll still spend quite a bit of money buying the 1970s parts. Here I have a table of the necessary old chips, their various names, and my best guess as to how much you'll pay (as seen by me on eBay USA at the time of publication). Some parts were only available from intel, while others were also available from National Semiconductor. Parts annotated with (g) are ceramic with gold (usually white ceramic, most rare, most expensive), (c) are parts that are grey ceramic (also rare, also expensive), and parts with no annotations are plastic and are the cheapest (still expensive). All of the part numbers listed in each cell function identically so you can buy the cheapest. I also designed this board to accept either a 4004 or a 4040, so you only need to buy one of them, probably whichever is cheapest. I am unable to explain the price difference between 4002-1 and 4002-2, since they are basically the same chip.

It should be noted that the prices are so high because of "collectors" who buy up these CPUs with no intention of ever using them. To them, any 16-pin chip laser-engraved with "P4004" would do just fine, but they instead insist on buying real chips, denying their use for real projects! What a dick move! However, it gets worse. The chips with gold caps are also sought out by people who "harvest" gold from them. This is a euphemism for grinding them up, destroying them forever. Ugh!

The modern parts

The modern parts of this project are downright affordable in comparison. The one thing that might be hard to source is the SPI VFD, but they do show up on eBay often, and I got mine for $15 each. Alternatively, another SPI display can be used with small code changes. You can also just not populate the VFD at all and interact with the device over the serial port only - this is fully supported and works well.

I also added some cleverness around the SPI PSRAM usage. PSRAM chip count and size is auto-detected. The first PSRAM must be at least 4MB in size, since the kernel expects to be loaded contiguously there and it is 2.5MB in size. So populate a 4MB or an 8MB chip first. If you work at ISSI and have access to a pre-production 16MB chip, that will also work. The second PSRAM chip can be left unpopulated, or populated with any size chip you have. I tested everything from 128KB to 8MB. Keep in mind that while more RAM will help Linux run better, it will slow down the boot process slightly. Sizes under 128KB are unsupported.

Thanks to my work on minimizing the size of the Linux kernel by aggressively culling the kernel config, the kernel I provide is small enough that you can boot to a shell prompt without using swap on only 4.5MB of RAM (eg: a 4MB chip + 512KB chip). You may then enable swap and go on. Although, given the cost of 8MB PSRAM chips, I do not know why you'd do this other than just curiosity.

How it works

The connections

SD card access

What could be simpler, one would think, than accessing an SD card? The spec is rather clear on how to do that over SPI: three wires to the card, one back; send some commands to init it, then one command is for read and another is for write. Trivial, right? It might be for modern microcontrollers that have kilobytes of RAM. The Linux/4004 board has a total of 440 bytes of RAM, if you count the un-addressable status nibbles, 352 if you do not. Of them, 160 (or 128 if you do not count the status nibbles) are hard-allocated to virtual MIPS registers, and another 160 (or 128 if you do not count the status nibbles) are hard-allocated to the TLB. This leaves 120 (96) bytes of RAM left. This is not enough to talk to an SD card, since the minimum unit of reading or writing an SD card is a 512-byte sector (very old SD cards allowed partial-sector reads, but partial-sector writes were never a thing). Adding another full RAM bank would only add 128 addressable bytes. It would take 4 full banks to fit a single 512-byte sector. This would force me to use an external 3-to-8 decoder to allow the 4004 to address this much memory. Plus, those chips are expensive! And, I had sworn I would not do this. Another solution was needed.

Well, the SD card has its own SPI bus, as do the PSRAMs. The emulator itself never really needs to process SD sectors' contents, only place them into the virtual RAM or write them from the virtual RAM to card. So, the SD sector data can always be read directly into PSRAM, or from it. This will work nicely since they have separate SPI busses. As we read card data, every 4 bytes we read, we'll write to PSRAM at the requested address, and then increment it by 4. Repeat 128 times. Writes will work much the same way. Due to the slowness of the 4004, this process is hilariously slow. It takes a bit over a second to read or write a sector to the card. But at least no extra 4002s chips are needed!

Which brings us to a potential issue with SD card access. The SD spec specifies that to properly initialize an SD card, one needs to send ACMD41 at a clock rate of 100KHz - 400KHz, and at least once every 50ms. The SPI bit-banged out the output ports of a 4002 cannot meet either of those timings. I cannot even approach them. I had some concerns. They went away quickly. Every SD card I tested happily initialized even at 5KHz, with ACMD41 sent every 200ms or even more rarely. I guess this makes sense since modern SD cards do not use the provided clock for anything internal, like original ones might have considered doing.

How it boots

The firmware necessarily must know how many TLB entries there are, since it needs to prevent the virtual MIPS CPU from populating the INDEX register with a higher value, and it needs to make sure that the RANDOM register also never presents a higher value. The firmware, thus, starts by probing the number of memory chips in the third bank, to then figure out how many TLB entries there are. Then, the VFD and the UART chip are initialized, then the SD card is initialized. If this fails, a message is shown: "Failed to init SD card. Halting here and now!". This message is the only string in the entire firmware, since bytes in this ROM are worth their weight in gold!

Every byte in the firmware has a high cost due to the very limited nature of how many bytes of ROM the 4004 can address. Additionally, having a virtual ROM requires an extra check in the decoding code for every memory access. No, it is not worth it to have a virtual ROM like the real DECstation 2100 had. Instead, my firmware simply loads the first SD card sector to the beginning of RAM (0x80000000), and then jumps to there. The code there is, as you'd expect, limited to 446 bytes that there is before the partition table starts. The code reads the partition table (which is also in RAM, as it is part of the first SD sector), finds a partition with type 0xBB, and reads it, in its entirety, to RAM at 0x80001000. If this goes well, it jumps there. If the partition is not found or the reads fail, a short message is shown using a trivial function that can output a string to the console using a hypercall to print each character.

How it runs

The art of it

The goal of this project was always partially artistic. This is why the board has a pretty VFD on it, why I designed it to look retro, and why it has hanging holes on the top corners! The idea is that, once completed, it can be hung on a wall where it will slowly do ... something. What could that something be? Well, I wrote a simple program that draws (in text mode) the mandelbrot set on the VFD (and on serial port). There are two versions: one uses floating point numbers (/root/mandelbrot in my image), which Linux ends up having to emulate, which makes it slow - around 30 days to draw the entire 13 rows x 40 cols image. The second version is integer-only (/root/mandelbrot_nofp in my image). That one completes in under 9 hours. But for the one I'll have hanging in my office, I have loftier goals. With swap enabled, the kernel sources can actually be built right on-device. It will take some number of years. The partition where the kernel lives is /dev/pvd2 and is mounted under /boot. The device can build its own kernel from source, copy it to /boot/vmlinux, and reboot into it. If power is interrupted, thanks to ext4, it will reboot, recover the filesystem damage from the journal, and restart the compilation process. That is my plan, at least.

I want one

Build one

The schematics are above, and the BOM of modern components is as follows. The table earlier summarizes the 1970s components needed. You can build one relatively easily.

Kit or pre-built

Making of the video

This was an adventure enough to warrant its own section

Capturing

Of course, this was not going to be easy, but I must admit that I severely underestimated how difficult this would end up being. The idea was simple: set up a phone to take a photo every second or so, collect the photos, make a video. Done. Right? Let's do some math. A 1920x1080 photo is around a megabyte in size. Taking a photo every two seconds generates around 1.76GB of data per hour. The filming took 9 days, which is 216 hours. It will, thus, produce around 379GB of photos in approximately 388 thousand files. I guess I'll need to come and offload all the photos from the device a few times to keep it from filling. Annoying but doable. Let's try!

Getting the data

Said hat tip was soon rescinded. SE3 does not have the storage to capture all the photos. I had planned to offload the data once a day using either the "Photos" app or the "Image Capture" app on a connected MacBook. After about two such offloads, Photos app stopped seeing new photos in device memory. I tried re-plugging USB and restarting the MacBook. Neither worked. Restarting the phone did, but of course this creates a gap in the recording and is unacceptable. Image Capture seemed to work better. It could copy photos off and delete them from the device, freeing space. Good! After three such offloads, the "delete" functionality stopped working. It could still copy new photos from the device but it refused to delete any. Again, the only thing that helped was restarting the phone. Again, this is unacceptable.

I tried a few more times, wasting a few more days, with no better luck. I gave up and grabbed an old 512GB iPhone 12 Pro Max with a shattered screen, shelled out $130 to have it replaced with the cheapest third-party display, and strapped it into the tripod. I decided to just capture everything nonstop and grab the photos later. This should have allowed me to skip the bugs in Photos and Image Capture apps.

Final photos taken, it was time to grab the images from the phone and stitch them into a video. This did not go well. Photos app only saw around 100,000 of them. The number differed every time I plugged the device in, but never rose to the correct number around 400,000. If I tried to tell it to import anyways, it hung. Leaving it alone for a day produced no results. Image Capture had a similar issue, although once it saw around 200,000 photos - a number that Photos never got up to. If told to import, it would say "Importing FILENAME.JPG" and stop there, having created a zero-sized file with the given name in the target directory.

OK, I can try to make a local backup of the iPhone to a MacBook, then decrypt the archive and get photos out, right? The backup button in Finder worked. As expected, the iPhone asked for the pincode to authorize the backup. The progress bar showed the moving cross-hatched pattern... a day later no progress had been made. Maybe AirDrop will work? Well, first of all, there is no way to select all photos in the Photos app (still) without dragging your finger across them all. You have no idea how long this takes for 400,000 photos, especially as the app gets slower and laggier as you go on. Somehow there is no "select all" option. I lost patience at 20,000 photos selected and decided to try to AirDrop those, then select more. The Photos app crashed trying to AirDrop this many photos. I tried with 10,000 and it crashed again.

Fine! I'll give up and use the cloud. I installed Google Photos and told it to backup to my account. It took over an hour "preparing to sync". It then synchronized around a hundred photos and promptly crashed. Opening it again produced the same result - over an hour of preparation, and another 100 photos got uploaded. This was not sustainable, neither in rate nor in manual labour required to keep re-opening the app. Unlike Apple's Photos, Google Photos DOES allow easy multi-select. With one tap I was able to select all photos taken on a given "Tuesday" and click the "Back up" icon. The app promptly crashed.

In frustration, I paid Apple for 2TB of iCloud, figuring that the cost of one month of it is fine to resolve my growing frustration. No crashes, and it started syncing. The sync speed hovered at around one photo every 5 seconds. My internet connection has 1 Gbit/s upload so it was not that problem. This was going to take forever!

Getting desperate

Maybe Windows could do better? I plugged the iPhone into my ThinkPad running Win10x64. It is supposed to show up in "My Computer" and have a DCIM directory with photos in it. It did the former, but not the latter. Googling around indicates that this is common - iPhone asks for pincode to confirm trusting the PC, and then nothing still. The typical "solutions" include "try cleaning your registry" and other such idiocy. OK... I guess it is Linux time? My Thinkpad had an install of Linux Mint 18 on it. It saw the iPhone but also saw no photos on it. Googling around indicated that some work had been done rather recently on iPhone Linux comms, and a tool called ifuse can successfully mount an iPhone using FUSE. The version in Linux Mint 18 was too old to work, of course. It failed with incomprehensible errors, and panicked my kernel once.

In the typical "it is called open source because you have to open the source to get anything at all to work" fashion, I had to download it and build it from source. Some dependencies were also too old, so I had to build them. Eventually, after much googling, building, installing, and cursing, all the things were built and it was time to try them. idevicepair successfully paired to the device. ifuse mounted the device at a given mountpoint, and there was indeed a non-empty DCIM directory in there! Success! I'll just drag that over to my desktop and be done, right? Well, that hung. I guess the Nautilus file manager was not prepared for such large copies. Maybe it tried to stat() each file or maybe it has some O(n^3) algorithms in there. I do not know. I re-plugged the iPhone, re-mounted it, and used good old cp -Rvf to copy all 400,000 files uneventfully ... over 10 hours.

Making the video

Dealing with this many files was annoying in a number of ways, and the sole working solution was almost always some soft of a CLI utility or script. After getting all the files out, they had to be renamed to sequential names. I did not dare mess with my hard-gotten source material, so instead they were copied with new names. A file list, vim, and one very gnarly regex later, a command list was created to do the copies. The copying took a little over 3 hours. I am not sure if I should blame ext4, md-raid, or myself for this...

I have never used any video editing apps, and I saw no reason to start now. A series of very messy ffmpeg commands were used to create the title screens, add the music to the first, create the video from the photos, and combine them all together into the final video. There is no time missing in the final video, but the playback speed is variable to speed up the boring bits. There is a clock and a calendar visible onscreen so you can easily see the time elapsed. Playback speed varies between 5FPS (10x realtime) and 960FPS (1920x realtime) to keep the video short(ish) and to-the-point(ish). It is hilarious that at some points in time you can see the changing of day to night and back by the light shining onto the table from the window (out of frame to the right). Due to the speed-up used, you may miss the blinking of the SD activity light (though you can see it blink a few times as the rootfs is being monted and later as ls executes). Sorry. However, if you really want to see it, see next paragraph.

Downloads

Credits

Comments...