Fundamentally they do the same thing, and are sort of just optimizations for particular problem-sets. A database is great for data that has proper indexes, a file system is great for much more arbitrary data [1].

If you’re a clever enough engineer, you can define a file system in terms of a database, as evidenced by iCloud. Personally, I have used this knowledge to use Cassandra to store blobs of video for HLS streams. This buys me a lot of Cassandra’s distributed niceties, at the cost of having to sort of reinvent some file system stuff.

[1] I realize that this is very simplified; I am just speaking extremely high level.

Conceptually that is quite true, though the domain dependencies make a lot of the code end up looking quite different.

But the first true database (pre-relational!) was developed for SABRE, American Airlines' computerized reservation system, in the early 1960s. Before that tickets were issued manually and the physical structure of the desks and filing systems used to make reservations reflected the need!

Unfortunately I can't find the paper I read (back in the mid 80s) on the SABRE database but I remember that record size (which is still used today!) was chosen based on the rotational speed of the disk and seek latency. Certainly there was no filesystem (the concept of filesystem barely existed, though Multics developed a hierarchical filesystem (intended to be quite database-like, as it happens) around the same time. The data base directly manipulated the disk. I don't know when that changed -- perhaps in the 1970s?

Like I said I can't quickly find the paper on the topic, but here's a nontechnical discussion with some cool pictures: https://www.sabre.com/files/Sabre-History.pdf. A search for "American Airlines SABRE database history" finds some interesting articles and a couple of good Wikipedia pages.

Some kinds of storage like cloud-scale object storage use custom HDD firmwares and custom on-disk formats instead of filesystems (±2005-era tech), we also have much newer solutions that do direct work on disks like HMR (not to be confused with HAMR or HAMMER2) where the host manages the recording of data on the disk. There are some generally available systems for that, but we also have articles like this: https://blog.westerndigital.com/host-managed-smr-dropbox/ (Which mostly focuses on SMR but this works on CMR too).

As for the record size in the DB vs. Disk attributes, that's probably not used like that anymore, but I do know that filesystem chunks/extents/blocks are calculated and grouped to profit from optimal LBA access. If you run ZFS and have it auto-detect or manually set the ashift size to make it match the actual on-disk sector size. This was especially relevant when 512e and 4Kn (and the various manufactures 'real' and 'soft' implementations) weren't reliable indicators of the best sector access size strategies.

I'm going to guess that by "domain dependency" you're talking about how

   handle = open("foo.txt");

   err = db->exec("SELECT * from DIRECTORY where NAME = 'foo.txt';", &result);

The best thing you can say about the way UNIX decided to handle files was that it forced people to either use them as is or make up their own scheme within a file itself (and don't get me started on the hell that is 'holey' files)

[1] In my case the underlying data storage was a NAND flash chip so the result you got back which was nominally a FILE* like stdio had the direct address on flash of where the bits were. read-modify-write operations were slow since it effectively copied the file for that (preserving flash sector write lifetimes)

But yes. Talking to mainframe people is a bit like talking to astronauts, in that their jargon is completely impenetrable to the uninitiated.

[1] https://en.wikipedia.org/wiki/IBM_2321_Data_Cell

Most likely showing a 24x80 3270 console session, with 8x16 character cells (if that much), but, still, quite awesome.

I'm not aware of any that ended up in a museum, sadly.

For those with sufficiently cool IEEE memberships, there is quite a lot about Sabre in the Annals of the History of Computing magazine archives.

https://ieeexplore.ieee.org/document/397059

https://ieeexplore.ieee.org/document/1114868

https://ieeexplore.ieee.org/document/279229, which is not about Sabre, but Air Canada's system.

If you think about it, modern IBM mainframes have a lot of weirdness about their filesystems and the concept of a file. Those machines are very alien for people who grew up on Unix.

Sounds like a record-oriented filesystem to me.

Which comes as no surprise as there is no difference between a database and a filesystem.

https://en.m.wikipedia.org/wiki/IBM_729

Having written an interface to FoundationDB in preparation to moving my app over to it, I couldn't disagree more.

Even "has proper indexes" is not something we'd agree on. In my case, for example, I am extremely happy with the fact that my indexes are computed in my app code, in my language, and that I am not restricted to some arbitrary concept of database-determined "indexable fields" and "indexable types".

Then there are correct transactions, versionstamps (for both keys and values), streaming large amounts of data, all of that in a distributed database, it's really nothing like a filesystem.

> has proper indexes

Does it matter where in the code the index lives? Are you arguing that databases don't have proper indexes or that filesystems don't? I'm not sure I'd agree with either argument.

> correct transactions

filesystems and databases have transactions, which one is "incorrect"?

> versionstamps (for both keys and values)

filesystems have timestamps, not sure what a versionstamp is but I suspect it's some domain specific name for a more general concept that both databases and filesystems utilize.

> streaming large amounts of data

many databases stream massive data and filesystems certainly do

> all of that in a distributed database

every major PaaS has some form of distributed filesystem

https://apple.github.io/foundationdb/data-modeling.html#vers...

For example if you have an Oracle DB, then it has a feature called DBFS that does this already:

https://docs.oracle.com/en/database/oracle/oracle-database/2...

You can instantiate a POSIX compatible FS using database tables, and then mount them using FUSE. From there you can export it via NFS if you wish. You can also export the FS via WebDAV and thus mount it over the network using the WebDAV support built in to Windows or macOS.

If you want to work with the FS transactionally, you have to do that using PL/SQL. POSIX doesn't define APIs for FS transactions, so some other approach is needed.

Because it's stored in the DB you can use all the other features of the RDBMS too like clustering, replication, encryption, compression and if need be, you can maintain indexes over the file content.

https://github.com/divyenduz/zoid-fs

Making a FUSE file system is sort of a bucket list thing I haven’t gotten around to doing yet. Maybe I should hack something together while I am still unemployed…

The BeOS filesystem was basically a database.

But there are a lot of differences between a database and a file system. A better way of thinking about it is that a filesystem is just a specialized database.

From an old school, a data base is really just a collection of data. An RDBMS = relational database. A filesystem is just another kind of database. etc etc.

It had what was basically a normal UNIX filing system, complete with /dev, /etc and so on, and it had support for indexing extended attributes. Your app was expected to create an index with a specific API at install time, and then after that writes to the indexed xattr would update a special "index directory". The OS could be given a simple query predicate with range and glob matching, and it would answer using the indexes, including a live query.

This was neat, but you could implement the same feature in Linux pretty easily. Nobody ever has, probably because xattrs historically didn't work that well. They don't get transmitted via common network protocols and have a history of getting lost when archiving, although I think these days every archive format in real use supports storing them.

There's also the question of how it interacts with POSIX file permissions. BeOS was an aggressively single user system so just didn't care. On Linux you'd need to think about how files that you can't read are treated in the indexing process.

Multiple devices also poses problems. BeOS simply required that apps create an index on a specific device themselves. If you plugged in a USB drive then files there just wouldn't show up in search unless the files had been created by not only BeOS, but an app you had previously installed. Note that installing an app post-hoc didn't work because creating an index didn't populate it with existing files, even if they had the right xattrs.

And of course it only worked with files. If you had content where the user's conception of a thing didn't map 1:1 to files, it was useless. For example you couldn't index elements within a document this way. Spotlight can index app states and screens, which also obviously BeOS couldn't do.

So there were a lot of limitations to this.

The modern equivalent would be writing a search plugin:

https://developer.apple.com/documentation/corespotlight/maki...

The API is more complex but lets you create search results that aren't directly tied to specific filing systems.

Aren't all databases specialized?

https://en.wikipedia.org/wiki/WinFS

> Cairo was the codename for a project at Microsoft from 1991 to 1996. Its charter was to build technologies for a next-generation operating system

https://en.wikipedia.org/wiki/Cairo_(operating_system)

In both cases, Longhorn and Cairo, the only thing that survived were a bunch of COM libraries.

But the distinction is pretty small. Both filesystems and databases are just wrappers around a data model. The former is primarily concerned with organizing data on a disk (with respect to space, speed, location and integrity), and the latter is primarily concerned with organizing and querying data (with respect to ease-of-use, speed and integrity).

People today seem to think relational databases were the first and only databases. But there many types of database: flat, hierarchical, dimensional, network, relational, entity–relationship, graph, object-oriented, object-relational, object-role, star, entity–attribute–value, navigational, document, time-series, semantic, and more.

The earliest filesystem, CP/M filesystem, was basically a flat database. Successive filesystems have taken on other data models, such as hierarchical, network and navigational. Since filesystems are used as a low-level interface to raw data, they didn't need more advanced data models or forms of query. On the other hand, IBM DB2, Hadoop, and Google File System are all forms of database filesystems, combining elements of both databases and filesystems.

It depends on how abstracted you're getting. I sometimes talk about the 30,000 foot view, but in this case, I might stretch the metaphor to say that from Low Earth Orbit, there is indeed not much difference between a database and a file system. In fact, there's not much difference between those things and some function calls. You put some parameters out, you get some stuff back.

From just slightly higher one realizes or remembers, it's all just numbers. You put some numbers into the system and get some other numbers out. Everything is built out of that.

You can build a database out of functions, a file system out of a database, functions out of a file system (albeit one beyond a blob store, think /proc or FUSE rather than ext2), you can mix network streams into any of these, anything you like.

And while it's helpful to be aware of that, at the same time, you are quite into architecture astronautics at that point and you are running low on metaphorical oxygen, and while the odd insight generated from this viewpoint might help here or there, if one wishes to actually build iCloud, one is going to have to come a great deal closer to Earth or one is going to fail.

Still, in the end, it's all just numbers in response to other numbers and the labels we humans put on exactly how the numbers are provided in response to other numbers are still the map and not the territory, even in the world of programming where arguably the map and the territory are as close as they can possibly be and still be in reality.

IIRC read only access worked well but I had issues getting write access working.

Stateless is basically moving everything to the back.

Im pretty sure google is doing the same thing/started with it.

Also this makes it 'easily' scalable horizontal: As soon as you are able to abstract on object level, you can scale your underlying infrastructure to just handle 'objects'.

⸻

1. I had read the original paper on database normalization over the summer and was on a database high at the time. I was young.

There was a nice video about how it is really hard to implement file systems because disks just don’t do what you expect.

Databases are a layer up and assume that they can at least write a blob somewhere and retrieve it with certain guarantees. Those guarantees are a thousand hacks in the file system implementation.

The more sophisticated the database kernel, the more the OS is treated like little more than a device driver.

And most journaling filesystems actually get in the way of databases which try to commit their own changelog to disk.

Many years back I came to the realiziation that a database is just a fancy data structure. I guess a file system is too.

It seems you are confusing database with database engine or possibly database management system. Querying is not a function of a database.

In fairness, the lazy speaker often says "database" in place of "database engine" and "database management system" to save on the energy of having to make additional sounds when one can infer what is meant by context, but in this case as "database" means database...

> (filesystems are rarely designed to facilitate them)

Facilitating querying is a primary objective of a filesystem database. What do you think "path/to/file.txt" is? That's right. It's a query!

File systems do not.

They have some similarity in that they both store data, but that's about it.

One can maybe argue that file systems are just an address book and databases are a more much complicated address book.

That was the promise of WinFS back in the day, which would have been really something had MS managed to bring it to fruition.

I still remember the hype from back then, in my opinion totally justified, too bad that things didn't come to be. I legit think that that project could have changed the face of computing as we know it today.

The issue with the filesystem-as-database concept is that unless you're doing it as a serverside thing to get RDBMS features for files, it doesn't give you much more power without very serious changes to applications.

The first problem is that databases are most useful when they index things, but files are just binary blobs in arbitrary formats. To index the contents you have to figure out what they are and parse them to derive interesting data. This is not best done by the filesystem itself though - you want it to be asynchronous, running in userspace and (these days) ideally sandboxed. This is expensive and so you don't want to do it on the critical file write path. Nowadays there are tools like Spotlight that do it this way and are useful enough.

If you don't do that then when it comes time to sell your shiny fs-as-a-db feature for upgrade dollars, you have to admit that your db doesn't actually index anything because no apps are changed to use it. Making them do so requires rewriting big parts from scratch. In that era I think the Office format was still essentially just memory dumps of internal data structures, done for efficiency, so making Office store documents as native database tables would have been a huge project and not yielded much benefit over simple text indexing using asynchronous plugins to a userspace search service.

Another problem is that databases aren't always great at indexing into the middle of blobs and changing them. Sometimes db engines want to copy values if you change them, because they're optimised for lots of tiny values (row column values) and not small numbers of huge values. But apps often want to write into the middle of files or append to them.

Yet another problem is that apps are very sensitive to filesystem performance (that's why the fs runs in the kernel to begin with). But databases do more work, so can be slower, which would make everything feel laggy.

So yeah it was a beautiful vision but it didn't work out. Note that operating systems started with databases as their native data storage mechanism in the mainframe era, and that was moved away from, because there are lots of things you want to store that aren't naturally database-y (images, videos, audio, source code etc).

https://learn.microsoft.com/en-us/sql/relational-databases/b...

It is, perhaps, possible another product with a different set of users with different needs could still find value in a relational filesystem, but Microsoft was unable to find that fit.

For example, instead of an OpenOffice document being a zip of XMLs, you'd just use a directory of XMLs, and then replace the XMLs with directories of tiny files for the attributes and node contents. Instead of a daemon having a config file, you'd just have a directory of tiny files. He claimed that apps weren't written that way already because filesystems were wasteful when files got too tiny.

Git is an example of a program that uses this technique, to some extent at least (modulo packfiles).

In reality, although that may have contributed, there are other reasons why people bundle data up into individual files. To disaggregate things (which is a good place to start if you want a filesystem-db merge) you also have to solve all those other reasons, which ReiserFS never did and as a project that "only" wanted to reinvent the FS, could not have solved.

Apple hit some of those issues when they tried making iLife documents be NeXT bundles:

1. Filesystem explorers treat files and directories differently for UI purposes. Apple solved it nicely by teaching the Finder to show bundle directories as if they were files unless you right click and select "Show contents". Or rather partly solved ... until you send data to friends using Windows, or Google Drive, or anything other than the Finder.

2. Network protocols like HTTP and MIME only understand files, not directories. In particular there is no standardised serialisation format for a directory beyond zip. Not solved. iLife migrated from bundles to a custom file format partly due to this problem, I think.

3. Operating systems provide much richer APIs for files than directories. You can monitor a file for changes, but if you want to monitor a directory tree, you have to iterate and do it yourself. You can lock a file against changes, but not a directory tree. You can check if a file has been modified by looking at its mtime, but there's no recursive mtime for directory trees. You can update files transactionally by writing to a temporary file and renaming, but you can't atomically replace a directory tree. Etc.

So the ReiserFS concept wasn't fully fleshed out, even if it had been accepted into the kernel. Our foundational APIs and protocols just aren't geared up for it. I've sometimes thought it'd be a neat retirement project one day to build an OS where files and directories are more closely merged as a concept, so files can have sub-files that you can browse into using 'cd' and so on, and those API/protocol gaps are closed. It wouldn't give you a full relational database but it'd be much more feasible to port apps to such an OS than to rewrite everything to use classical database APIs and semantics

Love when someone says something that makes my brain work!

For the most part you're spot on. HTTP has multipart messages that in theory could be extended to be composite of anything. So we could have those bundles! Oddly we can send to the server with a multipart message (forms)!!

I think that MIME is an interesting slice the OTHER way. You could store versions of the same document in a directory so HTML and JSON and XML OR a video or image in two formats and serve them up based on the MIME request.

Now if we could make one of those a multi part message...

And then how do you _download_ them? Browsers don't understand MIME at download time.

None of it is hard to solve. But, nobody ever did, and the value of doing things this new way is usually going to be lower than the value of smooth interop with everyone's different browser/OS/email/server combos.

TiDB is an example of a distributed SQL on top of a transactional key value store called TiKV.

Not sure why exactly it failed, I assume that it just wasn't a suitable idea at the time given that most consumer devices (especially laptops) had very slow traditional hard drives, but in the age of NVMe storage, maybe it would be worth revisiting, assuming that Microsoft is still interested in evolving Windows in meaningful ways outside of better Ad delivery mechanisms.

Much of the consumer-facing niceties of it got implemented in search tools that track metadata separately.

The only real downside is that the onramp for running FoundationDB at scale is quite a bit higher than a traditional distributed database.

When RecordLayer launched, I tested it out by building a catalog system that we could evolve and add new services with a single repository of protobuf schemas.

Now Apple is asking all file sync products like Dropbox to do the same, see Fileprovider API, breaking those as well. Really annoying

"FoundationDB: A Distributed Key-Value Store" [https://news.ycombinator.com/item?id=36572658]

"FoundationDB Record Layer" [https://news.ycombinator.com/item?id=18906341]

"Apple Acquires FoundationDB [https://news.ycombinator.com/item?id=9259986]

"How FoundationDB works and why it works" [https://news.ycombinator.com/item?id=37552085]

Obsidian is the only note app that I've stuck with. Notion/Apple Notes/Goodnotes/etc just had excessive pain points. Obsidian "just works" for my brain. Which is a relief, since the productivity app treadmill is exhausting.

[1] https://i.imgur.com/IrVy2mk.mp4

I'm utterly baffled why my iOS backups can live in Apple's cloud but not my Mac ones.

I honestly expected them to launch it years ago. The fact they still haven't seems to mean they've firmly decided not to for some reason, but I'm totally clueless as to what the reason could be.

Especially when making more money off services is a strategic priority for the company.

About every six months I do a fire drill and completely factory reset my macbook. Takes about 10 minutes for me to go from a fresh device to one that has all my apps, data, and developer tools ready to roll. Only annoying thing you can't really automate is signing into services like OneDrive or Dropbox, but this isn't a problem if you use iCloud Drive.

https://github.com/eh8/dotfiles

Redundancy because the thing most people care about backing up is media and important documents, which are likely already stored in iCloud. If you care about Time Machine back ups you probably want your whole filesystem with point-in-time restores. That's a lot more data for Apple to hang onto, for a small segment of its target market. Of course, Apple does have 2TB+ iCloud+ plans, but I would bet that the average iCloud+ subscriber is using nowhere near their limit.

Suddenly I find myself 1TB in, and desperate to find a fix!

On the other hand, that would require convincing your kids to do that...

But that's my point. To sell the 2TB plans to people who are merely on the free 5 GB or paid 50 GB plan.

And yes -- I don't even keep many files on my Mac, it's mostly in the cloud already. But if it gets lost/stolen, I want to restore all my apps and preferences the same way I do with my phone. Which is why I use Time Machine with a NAS, but it's silly to need a NAS at all. I just want to use the cloud.

The way I was looking at it is that Apple has successfully sold iCloud+ 2TB plans to a lot of people who don't need much more than 200GB. If everyone on the 2TB plan used even close to 2TB, I'd bet they'd have to charge me a lot more to make up the provisioning and usage costs of storage.

I would have no problem if Time Machine separated out OS and known signed application packages and basically just stored pointers to standard versions of them, as long as all that detection is done client-side.

There's no reason the backup would need to store anything but the list of those files (that list being encrypted), and then everything unique to me -- my configurations, my files, etc.

I'd bet that the rigid APIs on iOS also play a huge role here. Compared to the "anywhere you have permission to `open()` on disk" approach on macOS, iOS developers don't have as many options for where/how to store data. This probably makes backup / restore an order of magnitude simpler / reliable.

They’ve brought back SD slots in recent years: https://support.apple.com/en-us/102352

Meanwhile NVMes are a dime a dozen, and some laptops can fit two.

If anybody can decide to start building out massive datacenters of their own, it's Apple, and AWS/Azure know that.

And Apple just passes its rates along to the consumer. It costs what it costs.

In the HN guidelines, you read: "Otherwise, please use the original title, unless it is misleading or linkbait; don't editorialize."

I hope this will be taken into consideration.

Given that FoundationDB is built on top of SQLite, I wonder if that team is eyeing the HCTree engine for it.

It's still in experimental mode but provides literally 10x improvement on read/writes to SQLite.

Given Apple size & scale of iCloud, that seems like a massive win for them if that engine for SQLite can mature to production stability.

https://sqlite.org/hctree/doc/hctree/doc/hctree/threadtest.w...

Someone feel free to correct my memory if needed, I was not the primary person for this system or anything so I could be totally wrong.

Quite different from a SaaS-first approach where it actually makes sense to do "customer id column"-based multi-tenancy and one-migration-at-a-time schema changes that I think most of us at less-than-Apple scales are familiar with.

You can simply mirror the writes to both systems, and then migrate the old data underneath. As long as the data transfer preserves the cell level timestamps, the read path resolves any differences and compaction will eventually clean up any duplicates. (and sstable loads will have the timestamps)

Dynamodb does NOT have cell level timestamps, I believe they have row level timestamps. How it is doing globally replicated data and mutation merges: I have no idea. It seemed like a handwave when they were announcing it about two or three years ago.

1: https://dl.acm.org/doi/abs/10.14778/2536222.2536230

It is nothing you cannot overcome with the right hacks (what can't be overcome with the right hacks?), but it is frustrating that the network-based API wasn't designed for high latency use from the start. It didn't need to use the exact same API that was designed for a low-latency environment, but that's what we got. As SQL and web apps typically means MySQL or Postgres, that means you are apt to encounter the API design problems.

Granted, it seems there is renewed interest in SQLite to move the SQL database back to the way it was designed to be used. Which isn't surprising as all things in computing come and go in cycles. Once we round out that cycle and get back to "database on the network", maybe we can get a more well designed API meant for high latency to remove those frustrations.

This overview doesn't mention S3: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide...

Doh!

I'm planning to migrate to it. It's quite simply the best distributed database out there today.

https://stalw.art/docs/storage/backends/foundationdb

Phone says this since many years and iOS updates back: 10365 photos synced, Mac says: 10251 photos synced.

I only used that once, but it fixed all the months of odd syncing I had experienced.

Had the same problem and this fixed it.

[0] https://www.photosync-app.com/home

Perhaps it’s a Heisenberg type situation where measuring whether a file is synced itself changes the sync status.

Deleted messages would disappear in Big Sur. But read status wasn't synced for unread messages

In Catalina if you delete an unread message on iOS, it will disappear on MacOS, but Messages will still have an "1 message unread" badge.

2010 - We need 2 databases

2015 - We need 500 databases

2020 - We need billions of databases

2025 - Prediction: We need 1 database.