unsafe {
            let x_tmp0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x10);
            xmm_crc0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x01);
            xmm_crc1 = _mm_xor_si128(xmm_crc1, x_tmp0);
            xmm_crc1 = _mm_xor_si128(xmm_crc1, xmm_crc0);

Presumably with inline assembly both languages can emit what is effectively the same machine code. Is the Rust compiler a better optimizing compiler than C compilers?

In good practice it’s used judiciously in a codebase where it makes sense. Those sections receive extra attention and analysis by the developers.

Of course you can find sloppy codebases where people reach for unsafe as a way to get around Rust instead of writing code the Rust way, but that’s not the intent.

You can also find die-hard Rust users who think unsafe should never be used and make a point to avoid libraries that use it, but that’s excessive.

If you smell it when you're not working on the gas lines, that's a signal.

https://en.wikipedia.org/wiki/Odorizer#Natural_gas_odorizers

> Ethanethiol (EM), commonly known as ethyl mercaptan is used in liquefied petroleum gas (LPG) and resembles odor of leeks, onions, durian, or cooked cabbage

Methanethiol, commonly known as methyl mercaptan, is added to natural gas as an odorant, usually in mixtures containing methane. Its smell is reminiscent of rotten eggs or cabbage.

...but you can still call it "mercaptan" and be ~ correct in most cases.

Update its how the std lib does it: https://doc.rust-lang.org/src/alloc/collections/linked_list....

Coming up with these niche examples of things you need unsafe for in order to discredit rust’s safety guarantees is just not interesting. What fraction of programmer time is spent writing custom linked lists? Surely way less than 1%. In most of the other 99%, Rust is very helpful.

But it can occur naturally in natural gas.

https://en.m.wikipedia.org/wiki/Methanethiol

It's a bad comparison since CO doesn't smell, which is what makes it dangerous, while H2S is detected by our sense of smell at concentrations much lower than the toxic dose (in fact, its biggest dangers comes from the fact that at dangerous concentration it doesn't even smell anything due to our receptors being saturated).

It's not what's being put in natural gas, but it wouldn't be that dangerous if we did.

I am 100% sure that the smell they add to natural gas does not smell like rotten eggs.

No need to get all moral about it.

In fact, it has already been merged two weeks ago: https://github.com/rust-lang/stdarch/pull/1714

The change is already visible on nightly: https://doc.rust-lang.org/nightly/core/arch/x86/fn._mm_xor_s...

Compared to stable: https://doc.rust-lang.org/core/arch/x86/fn._mm_xor_si128.htm...

So this should be stable in 1.87 on May 15 (Rust's 10 year anniversary since 1.0)

There was a bug open about it and the rationale was that no one with the expertise (some of these are quite arcane) was stepping up to do it. (edit: other comments in this thread suggest that this effort is now underway and first changes were committed a few weeks ago)

You can do safe SIMD using std::simd but it is nightly only at this point.

[1] https://doc.rust-lang.org/std/simd/index.html

Of course, in practice, even in Rust, it isn't strictly true that programs without unsafe can't crash with fatal runtime errors. There's always stack overflows, which will crash you with a SIGABRT or equivalent operating system error.

https://doc.rust-lang.org/std/intrinsics/simd/index.html

So I suspect it's a matter of two things:

1. You're calling out to what's basically assembly, so buyer beware. This is basically FFI into C/asm.

2. There's no guarantee on what comes out of those 128-bit vectors after to follow any sanity or expectations, so... buyer beware. Same reason std::mem::transmute is marked unsafe.

It's really the weakest form of unsafe.

Still entirely within the bounds of a sane person to reason about.

For example, it is perfectly legal to dereference a vector pointer that references illegal memory if you mask the illegal addresses. This is a useful trick and common in e.g. idiomatic AVX-512 code. The mask registers are almost always computed at runtime so it would be effectively impossible to determine if a potentially illegal dereference is actually illegal at compile-time.

I suspect we’ll be hand-rolling unsafe SIMD for a long time. The different ISAs are too different, inconsistent, and weird. A compiler that could make this clean and safe is like fusion power, it has always been 10 years away my entire career.

Of safe SIMD, but some stuff in core::arch is stabilized. Here's the first bit called in the example of the OP: https://doc.rust-lang.org/core/arch/x86/fn._mm_clmulepi64_si...

When I started working in Rust, I'd want some feature or function, look it up, and find it was unstable, sometimes for years. This was frustrating at first, but then I'd go read the GitHub issue thread and find that there was some design or implementation concern that needed to be overcome, and that people were actively working on it and unwilling to stabilize the feature until they were sure it was the best possible design. And the result of that is that features that do get stabilized are well thought out, generalize, and compose well with everything else in the language.

Yes, I really want things like portable SIMD, allocators, generators, or Iterator::intersperse. But programming languages are the one place I really do want perfect to be the enemy of good. I'd rather it take 5+ years to stabilize features than for us to end up with another Swift or C++.

Sounds like the Rust's async story.

Rust uses this strategy of minimal/incremental stabilization quite often (see also: const generics, impl Trait); the difference between this and what drove me away from Swift is that MVPs aren't shipped unless it's clear that the design choices being made now will still be the right choices when the rest of the feature is ready.

I would argue that it's the opposite of a mistake. If you standardize everything before the ecosystem gets a chance to play with it, you risk making mistakes that you have to live with in perpetuity.

Language designers admittedly should worry about constant breakage but it’s fine to have some churn, and we shouldn’t be so concerned of it that it freezes everything

In nightly?

Hard disagree. Letting people try things out in the real world is how you avoid half-baked features. Easy availability of nightly compilers with unstable features allows way more people to get involved in the pre-stabilization polishing phase of things and raise practical concerns instead of theoretical ones.

C++ takes the approach of writing and nitpicking whitepapers for years before any implementations are ready and it's hard to see how that has led to better outcomes relatively speaking.

1. Big library/compiler does a thing, and people really like it

2. Other compilers and libraries copy that thing, sometimes putting their own spin on it

3. All the kinks get worked out and they write a white paper

4. Eventually the thing becomes standard

That way, everything in the standard library is something that is fully-thought-out and feature-complete. It also gives much more room for competing implementations to be built and considered before someone stakes out a spot in the standard library for their thing.

Fully-thought-out and feature-complete is something that since C++17 has been hardly happening.

Are C++ features really that much better thought out? Modules were "standardized" half a decade ago, but the list of problems with actually using them in practice is still pretty damn long to the point where adoption is basically non-existent.

I'm not going to pretend to be nearly as knowledgeable about C++ as Rust, but it seems like most new C++ features I hear about are a bit janky or don't actually fit that well with the rest of the language. Something that tends to happen when designing things in an ivory tower without testing them in practice.

The fundamental problem with modules is that build systems for C++ have different abstractions and boundaries. C++ modules are like Rust async - something that just doesn't fit well with the language/system and got hammered in anyway.

The reason it seems like they come from nowhere is probably because you don't know where they come from. Most things go through boost, folly, absl, clang, or GCC (or are vendor-specific features) before going to std.

That being said, it's not just C++ that has this flow for adding features to the language. Almost every other major language that is not Rust has an authoritative specification.

This take makes me sad. There are a lot of reasons why an open source contributor may not see something through. "Lack of discipline" is only one of them. Others that come to mind are: lack of time, lack of resources, lack of capability (i.e good at writing code, but struggles to navigate the social complexities of sheparding a significant code change), clinically impaired ability to "stay the course" and "see things through" (e.g. ADHD), or maybe it was a collaborative effort and some of the parties dropped out for any of the aforementioned reasons.

I don't have a solution, but it does kinda suck that open source contribution processes are so dependent on instigators being the responsible party to seeing a change all the way through the pipeline.

I don't have the personality or time to wade into committee type work, so I have no idea what it would take to get those two across the finish line, but the allocator one in particular makes me question Rust for lower level applications. I think it's just not going to happen.

If Zig had proper ADTs and something equivalent to borrow checker, I'd be inclined to poke at it more.

"People are fallible, and mistakes will happen, but by requiring these five unsafe operations to be inside blocks annotated with unsafe you’ll know that any errors related to memory safety must be within an unsafe block. Keep unsafe blocks small; you’ll be thankful later when you investigate memory bugs."

I hope the SIMD intrinsics make it to stable soon so folks can ditch unnecessary unsafes if that's the only issue.

If you want a clean crash instead on indeterministic behavior, you need to use assert like in C, but it won't save you from compiler optimization removing checks that are deemed useless (again, exactly like in C).

People seem to disagree.

Unsafe Rust Is Harder Than C

https://chadaustin.me/2024/10/intrusive-linked-list-in-rust/

https://news.ycombinator.com/item?id=41944121

It's like letting a wet dog (who'd just been swimming in a nearby swamp) run loose inside your hermetically sealed cleanroom.

"You can take five actions in unsafe Rust that you can’t in safe Rust, which we call unsafe superpowers. Those superpowers include the ability to:

    Dereference a raw pointer
    Call an unsafe function or method
    Access or modify a mutable static variable
    Implement an unsafe trait
    Access fields of a union

In addition, unsafe does not mean the code inside the block is necessarily dangerous or that it will definitely have memory safety problems: the intent is that as the programmer, you’ll ensure the code inside an unsafe block will access memory in a valid way.

People are fallible, and mistakes will happen, but by requiring these five unsafe operations to be inside blocks annotated with unsafe you’ll know that any errors related to memory safety must be within an unsafe block. Keep unsafe blocks small; you’ll be thankful later when you investigate memory bugs."

But using ordinary module encapsulation and private fields, you can scope the code that needs to uphold those preconditions to a particular module.

So the "trusted computing base" for the unsafe code can still be scoped and limited, allowing you to reduce the amount of code you need to audit and be particularly careful about for upholding safety guarantees.

Basically, when writing unsafe code, the actual unsafe operations are scoped to only the unsafe blocks, and they have preconditions that you need to scope to a particular module boundary to ensure that there's a limited amount of code that needs to be audited to ensure it upholds all of the safety invariants.

Ralf Jung has written a number of good papers and blog posts on this topic.

And even if you try to provide some kind of safer abstraction, you're limited by the much more primitive type system, that can't distinguish between owned types, unique borrows, and shared borrows, nor can it distinguish thread safety properties.

So you're left to convention and documentation for that kind of information, but nothing checking that you're getting it right, making it easy to make mistakes. And even if you get it right at first, a refactor could change your invariants, and without a type system enforcing them, you never know until someone comes along with a fuzzer and figures out that they can pwn you

   int average(int x, int y) {
       return (x+y)/2;
   }

I might be old, but more than 10 years ago, hardly anyone talked about UB in C and C++ programming. In the last 10 years, it is all the rage, but seems to add very little to the conversation. For example, if you program C or C++ with the Win32 API, there are loads of weird UB-ish things that seem to work fine.

This is not how compilers work. Optimization happens based on language semantics, not on what platforms do.

UB in C is often found where different real hardware architectures had incompatible behavior. Rather than biasing the language for or against different architectures they left it to the compiler to figure out how to optimize for the cases where instruction behavior diverge. This is still true on current architectures e.g. shift overflow behavior which is why shift overflow is UB.

  int average(int x, int y) {
    long sum = (long)x + y;
    if(sum > INT_MAX || sum < INT_MIN)
        return -1; // or any value that indicates an error/overflow
  
    return (int)(sum / 2);
  }

There is no guarantee that sizeof(long) > sizeof(int), in fact the GNU libc documentation states that int and long have the same size on the majority of supported platforms.

https://www.gnu.org/software/libc/manual/html_node/Range-of-...

> return -1; // or any value that indicates an error/overflow

-1 is a perfectly valid average for various inputs. You could return the larger type to encode an error value that is not a valid output or just output the error and average in two distinct variables.

AI and C seem like a match made in hell.

C libraries tend to support a lot of exotic platforms. zlib for example supports Unicos, where int, long int and long long int are all 64 bits large.

- Exposing an unsafe API and relying on the caller to manually uphold invariants

- Doing things like defensive copying at a performance cost

In many cases Rust gives you the best of both worlds: sharing memory liberally while still having the compiler enforce correctness.

In practice (in both languages) you check what the actual unsafe code does (or "all" code in C's case), note code that depends on external actors for safety (it's not all C code, nor is it all unsafe Rust blocks), and check their callers (and callers callers, etc).

Rust's type system (including ownership and borrowing, Sync/Send, etc), along with it's privacy features (allowing types to have private fields that can only be accessed by code in the module that defined them) allows you to create fully safe interfaces around code that uses unsafe; there is provably no combination of uses of the interface which lead to undefined behavior.

Now, yeah, it's possible to also use unsafe in Rust just for applying a local optimisation. And that has fewer benefits than a fully encapsulated safe interface, though is still easier to audit for potential UB than C.

So you're right that it's on a continuum, but the distinction between safe and unsafe code means you can more easily find the specific places where UB could occur, and the encapsulation and type system makes it possible to create safe abstractions over unsafe code.

From my past experiences with Rust, the team never had to think about data race once, or mutable volatile globals. And we all there suffered from those decades ago with C and sometimes C++ as well.

You like those and don't want to migrate? More power to ya! But badmouthing Rust with what seem fairly uninformed comments is just low. Inform yourself first.

That's only true at the same level of scrutiny as "all C operations can cause undefined behaviour, regardless of what they are", which I find similarly shallow.

And there is not many things we have statistics on in CS, but memory vulnerabilities being absolutely everywhere in unsafe languages, and Rust cleaning up the absolute majority of them even when only the new parts are written in Rust are some of the few we do know, based on actual, real life projects at Google/Microsoft among others.

A memory safe low-level language is as novel as it gets. Rust is absolutely not just hype, it actually delivers and you might want to get on with the times.

    > absolutely laughable c-strings that perform terribly

Sure, you can technically just write your own vulnerability for your own program and inject it at an unsafe and see the whole world crumble... but the exact same is true for any form of FFI calls in any language. Is Java memory safe? Yeah, just because I can grab a random pointer and technically break anything I want won't change that.

The fact that a memory vulnerability error may either appear at no place at all OR at the couple hundred lines of code thorough the whole project is a night and day difference.

So, in theory, unsafe rust opens the floodgates. In practice, though, you can use small fragments of unsafe code that programmers can fairly easily check to be safe.

Then, once you’ve convinced yourself that those fragments are safe, you can be assured that your whole program is safe (using ‘safe’ in the rust sense, of course)

So, there may be some small islands of unsafe code that require extra attention from the programmer, but that should be just a tiny fraction of all lines, and you should be able to verify those islands in isolation.

This is where the rubber hits the road. Rust does not allow you to do this, in the sense that this is possibly undefined behavior. That "possibly" is why the compiler allows you to write this code, because by saying "unsafe", you are promising that this specific arbitrary address is legal for you to write to. But that doesn't mean that it's always legal to do so.

A simple example might be modifying a const value deep down in some class, where it only becomes apparent later in the program’s execution. Hence their analogy of the wet dog in a clean room - whatever beliefs you have about the structure of memory in your entire program, and guaranteed by the compiler, could have been undone by a rogue unsafe.

Rust encourages using unsafe to "teach" the language new design patterns and data structures; and uses this heavily in its standard library. For example, the Vec type is a wrapper around a raw pointer, length, and capacity; and exposes a safe interface allowing you to create, manipulate, and access vectors with no risk of pointer math going wrong -- assuming the people who implemented the unsafe code inside of Vec didn't make a mistake, the external, safe interface is guaranteed to be sound no matter what external code does.

Think of unsafe not as "this code is unsafe", but as "I've proven this code to be safe, and the borrow checker can rely on it to prove the safety of the rest of my program."

    > For example, the Vec type is a wrapper around a raw pointer, length, and capacity; and exposes a safe interface allowing you to create, manipulate, and access vectors with no risk of pointer math going wrong -- assuming the people who implemented the unsafe code inside of Vec didn't make a mistake, the external, safe interface is guaranteed to be sound no matter what external code does.

Instead, all that functionality is written as Rust code in the standard library, such as Vec. This is what I mean by using unsafe code to "teach" the borrow checker: the language itself doesn't have any notion of growable arrays, so you use unsafe to define its semantics and interface, and now the borrow checker understands growable arrays. The alternative would be to make growable arrays some kind of compiler magic, but that's both harder to implement correctly and not generalizable.

> you can do exactly the same in C by using an opaque pointer to protect the data structure. Then you write a bunch of functions that operate on the opaque pointer. You can use assert() to protect against unreasonable inputs.

That's true and that's a great design pattern in C as well. But there are some crucial differences:

- Rust has no undefined behavior outside of unsafe blocks. This means you only need to audit unsafe blocks (and any invariants they assume) to be sure your program is UB-free. C does not have this property even if you code defensively at interface boundaries.

- In Rust, most of the invariants can be checked at compile time; the need for runtime asserts is less than in C.

- C provides no way to defend against dangling pointers without additional tooling & runtime overhead. For instance, if I write a dynamic vector and get a pointer to the element, there's no way to prevent me from using that pointer after I've freed the vector, or appended an element causing the container to get reallocated elsewhere.

Rust isn't some kind of silver bullet where you feed it C-like code and out comes memory safety. It's also not some kind of high-overhead garbage collected language where you have to write unsafe whenever you care about performance. Rather, Rust's philosophy is to allow you to define fundamental operations out of small encapsulated unsafe building blocks, and its magic is in being able to prove that the composition of these operations is safe, given the soundness of the individual components.

The stdlib provides enough of these building blocks for almost everything you need to do. Unsafe code in library/systems code is rare and used to teach the language of new patterns or data structures that can't be expressed solely in terms of the types exposed by the stdlib. Unsafe in application-level code is virtually never necessary.

By the way, the rust compiler does generate such code because under the hood LLVM runs an autovectorizer when you turn on optimizations. However, for the autovectorizer to do a good job you have to write code in a very special way and you have no way of controlling whether or not it kicked in and once it did that it did a good job.

There’s work on creating safe abstractions (that also transparently scale to the appropriate vector instruction), but progress on that has felt slow to me personally and it’s not available outside nightly currently.

    > However, for the autovectorizer to do a good job you have to write code in a very special way

There's even unsafe usage in the standard library and it's used a lot in embedded libraries.

Sorry but horrible comparison ;)

If you need to rely on unsafe in a memory-safe language for performance reasons, then there is a issue with the language compiler at that point, that needs to be fixed. Simple as that.

The whole memory-safety is the bread and butter of the language, the moment you start to bypass it for faster memory operations, you can start doing the same in any other language. I mean, your literally bypassing the main selling point of the language. \_00_/

Once you can internalize that you could unlock the power of encapsulation.

All safe code in existence running on von Neumann architectures is built on a foundation of unsafe code. The goal of all memory-safe languages is to provide safe abstractions on top of an unsafe core.

    > What language is the JVM written in?

I like your second paragraph. It is well written.

- Java 74.1%

- C++ 14.0%

- C 7.9%

- Assembly 2.7%

And those values have been increasing for Java with each OpenJDK release.

This isn't a wet dog in a cleanroom. This is cleanroom complex that has a very small outhouse that is labeled as dangerous.

At the same time, unsafe doesn't just turn off all compiler checks, it just gives you tools to go around them, as well as tools that happen to go around them because of the way they work. Rust unsafe is this weird mix of being safer than pure C, but harder to grasp; with lots of nuanced invariants you have to uphold. If you want to ensure your code still has all the nice properties the compiler guarantees (which go way beyond memory safety) you would have to carefully examine every unsafe block. Which few people do, but you generally still end up with a better status quo than C/C++ where any code can in principle break properties other code was trying to uphold.

As soon as you start playing with FFI and raw pointers in Python, NodeJS, Julia, R, C#, etc you can easily loose the nice memory-safety properties of those languages - create undefined behavior, segfaults, etc. I'd say Rust is a lot nicer for checking unsafe correctness than other memory-safe languages, and also makes it easier to dip down to systems-level programming, yet it seems to get a lot of hate for these features.

Inside that block, both yes and no. You have to enforce those nice guarantees yourself. Code that violates it will still crash.

[1]: https://www.youtube.com/watch?v=rTo2u13lVcQ

Normally in safe code you can’t violate the language rules because the compiler enforces various rules. In unsafe mode, you can do several things the compiler would normally prevent you from doing (e.g. dereferencing a naked pointer). If you uphold all the preconditions of the language, safety is preserved.

What’s unfortunate is that the rules you are required to uphold can be more complex than you might anticipate if you’re trying to use unsafe to write C-like code. What’s fortunate is that you rarely need to do this in normal code and in SIMD which is what the snippet is representing there’s not much danger of violating the rules.

To continue the analogy of the dog, you let the dog get wet (=you use unsafe), but you put a cleaning room (=the sound and safe API) before your sealed room (=the safe code world)

No, not even close. You only lose Rust's safety guarantees when your unsafe code causes Undefined Behavior. Unsafe code that can be made to cause UB from Safe Rust is typically called unsound, and unsafe code that cannot be made to cause UB from Safe Rust is called sound. As long as your unsafe code is sound, then it does not break any of Rust's guarantees.

For example, unsafe code can still use slices or references provided by Safe Rust, because those are always guaranteed to be valid, even in an unsafe block. However, if from inside that unsafe block you then go on to manufacture an invalid slice or reference using unsafe functions, that is UB and you lose Rust's safety guarantees because of the UB.

Not sure why would one resulted in all. One of Rust's advantages is the clear boundary between safe/unsafe.

In other words, unsafe works if you use it carefully and keep it contained.

O_o

It tends to be found in drivers, kernels, vector code, and low-level implementations of data structures and allocators and similar things. Not typical application code.

As a general rule it should be avoided unless there's a good reason to do it. But it's there for a reason. It's almost impossible to create a systems language that imposes any kind of rules (like ownership etc.) that covers all possible cases and all possible optimization patterns on all hardware.

Unsafe Rust is currently extremely underspecified and underdocumented, but it's designed to be far more specifiable than C. For example: aliasing rules. When and how you're allowed to alias references in unsafe code is not at all documented and under much active discussion; whereas in C pointer aliasing rules are well defined but also completely insane (casting pointers to a different type in order to reinterpret the bytes of an object is often UB even in completely innocuous cases).

Once Rust's memory model is fully specified and written down, unsafe Rust is trying to go for something much simpler, more teachable, and with less footguns than C.

Huge props to Ralf Jung and the opsem team who are working on answering these questions & creating a formal specification: https://github.com/rust-lang/unsafe-code-guidelines/issues

All languages at some point interface with syscalls or low level assembly that can be done wrong, but one of Rust's selling points is a safe wrapping of low-level interactions. Like safe heap allocation/deallocation with `Box`, or swapping with `swap`, etc. Except... here.

Why does a library like zlib need to go beyond Rust's safe offerings? Why doesn't rust provide safe versions of the constructs zlib needs?

Some codebases, you can grep for "unsafe", find no results, and conclude the codebase is safe... if you trust its dependencies.

This is not one of those codebases. This one uses unsafe liberally, which tells you it's about as safe as C.

"unsafe behaviour is clearly marked" seems to be a thought-stopping cliche in the Rust world. What's the point of marking them, if you still have them? If every pointer dereference in C code had to be marked unsafe (or "please" like in Intercal), that wouldn't make C any better.

rustc uses LLVM just as clang does, so to a first approximation they're the same. For any given LLVM IR you can mostly write equivalent Rust and C++ that causes the respective compiler to emit it (the switch fallthrough thing mentioned in the article is interesting though!) So if you're talking about what's possible (as opposed to what's idiomatic), the question of "which language is faster" isn't very interesting.

Rust zlib is faster than zlib-ng, but the latter isn't a particularly fast C contender. Chrome ships a faster C zlib library which Rust could not beat.

Rust beat C by using pre-optimized code paths and then C function pointers inside unsafe. Plus C SIMD inside unsafe.

I'd summarize the article as: generous chunks of C embedded into unsafe blocks help Rust to be almost as fast as Chrome's C Zlib.

Yay! Rust sure showed it's superiority here!!!!1!1111

There was Ispc, which was a separate C-like programming language just for SIMD, but I don't understand why can't regular compilers generated high-quality vectorized code.

Which is exactly the point, other languages have unsafe implicitly sprinkled in every single line.

Rust tries to bound and explicitly delimit where unsafe code is to makes review and verification efforts precise.

  In addition, unsafe does not mean the code inside the
  block is necessarily dangerous or that it will definitely
  have memory safety problems: the intent is that as the
  programmer, you’ll ensure the code inside an unsafe block
  will access memory in a valid way.

Bad C programmers though? Their stuff is more dangerous and they don't know when and don't call it out and should probably stick to Rust.

I take a hard line on this stuff because we can either keep repeating the fundamental mistake of believing things like "willpower" to write correct code are real, or we can move on and adopt better tooling.

People can write memory safe code, just not 100% of the time.

In safe Rust (the default), memory access is validated by the borrow checker and type system. Rust’s goal of soundness means safe Rust should never cause out-of-bounds access, use-after-free, etc; if it does, then there's a bug in the Rust compiler.

> The standard library will not deviate in naming or type signature of any intrinsic defined by an architecture.

I think this makes sense, just like any other intrinsic: unsafe to use directly, but with safe wrappers.

I believe that there are also some SIMD things that would have to inherently take raw pointers, as they work on pointers that aren't aligned, and/or otherwise not valid for references. In theory you could make only those take raw pointers, but I think the blanket policy of "follow upstream" is more important.

Is that not true? Even validators have bugs or miss things no?

Until we design perfectly correct computer hardware, processors, and a sun which doesn't produce solar radiation, we can't rely on totally uniform correct execution of our code, so we should give up.

The reality is that while we can't prove the rust compiler is safe, we can keep using it and diligently fix any counter-examples, and that's good enough in practice. Over in the real world, where we can acknowledge "yes, it is impossible to prove the absence of all bugs" and simultaneously say "but things sure seem to be working great, so we can get on with life and fix em if/when they pop up".

If you take Rust at face value, than this to me seems like an obvious question to ask

Like, when I say "use signal, it's secure", someone could respond "Ahh, but technically you can't prove the absence of bugs, signal could have serious bugs, so it's not secure, you fool", but like everyone reading this already knew "it's secure" means "based on current evidence and my opinion it seems likely to be more secure than alternatives", and it got shortened. Interpreting things as absolutes that are true or false is pointless debate-bro junk which lets you create strawmen out of normal human speech.

When someone says "1+1 = 2", and a debate-bro responds "ahh but in base-2 it's 10 you fool", it's just useless internet noise. Sure, it's correct, but it's irrelevant, everyone already knows it, the original comment didn't mean otherwise.

Responding to "safe Rust should never cause out-of-bounds access, use-after-free" with "ahh but we can't prove the compiler is safe, so rust isn't safe is it??" is a similarly sorta response. Everyone already knows it. It's self-evident. It adds nothing. It sounds like debate-bro "I want to argue with you so I'm saying something that's true, but we both already know and doesn't actually matter".

I think that allergic response came out, apologies if it was misguided in this case and you're not being a debate-bro.

Yep! For example, https://github.com/Speykious/cve-rs is an example of a bug in the Rust compiler, which allows something that it shouldn't. It's on its way to being fixed.

> or miss things no?

This is the trickier part! Yes, even proofs have axioms, that is, things that are accepted without proof, that the rest of the proof is built on top of. If an axiom is incorrect, so is the proof, even though we've proven it.

The Rust compiler is indeed better than the C one, largely because of having more information and doing full-program optimisation. A `vec_foo = vec_foo.into_iter().map(...).collect::Vec`, for example, isn't going to do any bounds checks or allocate.

> isn't going to do any bounds checks or allocate.

You need to add explicit bounds check or explicitly allocate in C though. It is not there if you do not add it yourself.

However, if you verify that the unsafe blocks are correct, and the safe API wrapping them rejects invalid inputs, then they won't be able to cause unsafety anywhere.

This does reduce how much code you need to review for memory safety issues. Once it's encapsulated in a safe API, the compiler ensures it can't be broken.

This encapsulation also prevents combinatorial explosion of complexity when multiple (unsafe) libraries interact.

I can take zlib-rs, and some multi-threaded job executor (also unsafe internally), but I don't need to specifically check how these two interact. zlib-rs needs to ensure they use slices and lifetimes correctly, the threading library needs to ensure it uses correct lifetimes and type bounds, and then the compiler will check all interactions between these two libraries for me. That's like (M+N) complexity to deal with instead of (M*N).

It's due to a couple of different things interacting with each other: unsafe relies on invariants that safe code must also uphold, and that the privacy boundary in Rust is the module.

Before we get into the unsafe stuff, I want you to consider an example. Is this Rust code okay?

    struct Foo {
       bar: usize,
    }
    
    impl Foo {
        fn set_bar(&mut self, bar: usize) {
            self.bar = bar;
        }
    }

Let's talk about unsafe. The canonical example of unsafe code being affected outside of unsafe itself is the implementation of Vec. Vecs look something like this (the real code is different for reasons that don't really matter in this context):

    struct Vec {
       ptr: *mut T,
       len: usize,
       cap: usize,
    }

That property is very important! If the length is greater than the capacity, when we try and index into the Vec, we'd be accessing random memory.

So now, this function, which is the same as Foo::set_bar, is no longer okay:

    impl Vec {
        fn set_len(&mut self, len: usize) {
            self.len = len;
        }
    }

Poorly-written unsafe code can have effects extending out into safe code. But correctly-written unsafe code does not have any effects on safe code w.r.t. memory safety. So to ensure memory safety, you just have to verify the correctness of the unsafe code (and any helper functions, etc., it depends on), rather than the entire codebase.

Also, some forms of unsafe code are far less dangeous than others in practice. E.g., most of the SIMD functions are practically safe to call in every situation, but they all have 'unsafe' slapped on them due to being intrinsics.

> You need to add explicit bounds check or explicitly allocate in C though. It is not there if you do not add it yourself.

Unfortunately, you do need to allocate a new buffer in C if you change the type of the elements. The annoying side of strict aliasing is that every buffer has a single type that's set in stone for all time. (Unless you preemptively use unions for everything.)

I think the bigger point here is that doing SIMD in Rust is still painful.

There are efforts like portable-simd [1] to make this better, but in practice, many people are dropping down to low-level SIMD intrinsics and/or inline assembly, which are no better than their C equivalents.

[1]: https://github.com/rust-lang/portable-simd

This is such a widespread misunderstanding… one of the points of rust (there are many other advantages that have nothing to do with safety, but let’s ignore those for now) is that you can build safe interfaces, possibly on top of unsafe code. It’s not that all code is magically safe all the time.

Unsafe code is not inherently faster than safe code, though sometimes, it is. Unsafe is for when you want to do something that is legal, but the compiler cannot understand that it is legal.

Assembly language faster than C. And faster than Rust. Assembly can be very fast.

What wrong with that?

That being said, most rust programs don't ever need to use unsafe directly. If you go very low level or tune for prrformance it might become useful however.

Or if you're lazy and just want to stop the borrow checker from saving your ass.

..at least outside of loads/stores. From a bit of looking at the code though it seems like a good amount of those should be doable in a safe way with some abstractions.

That depends. If, for you, safety is something relative and imperfect rather than absolute, guaranteed and reliable, then - the answer is that once you have the first non-trivial unsafe block that has not gotten standard-library-level of scrutiny. But if that's your view, you should not be all that starry-eyed about how "Rust is a safe language!" to begin with.

On the other hand, if you really do want to rely on Rust's strong safety guarantees, then the answer is: From the moment you use any library with unsafe code.

My 2 cents, anyway.

We will see more and more Rust libraries trounce their C counterparts in speed, because Rust is more fun to work in because of the above. Rust has democratized high-speed and concurrent systems programming. Projects in it will attract a larger, more diverse developer base -- developers who would be loath to touch a C code base for (very justified) fear of breaking something.

But even then, your code is calling out to kernel functions which are probably written in C or assembly, and therefore "dangerous."

Rust code safety is overhyped frequently, but reducing an attack surface is still an improvement over not doing so.

Well, good thing that nobody sane is saying that then.

If I read TFA correctly, they came up with a library that is API compatible with the C one, but they've measured to be faster.

At that point I think in addition to safety benefits in other parts of the library (apart from unsafe micro optimizations as quoted), what they're leveraging is better compiler technology. Intuitively, I start to assume that the rust compiler can perhaps get away with more optimizations that might not be safe to assume in C.

Perhaps it is faster than already-existing implementations, sure, but not "faster than C", and it is odd to make such claims.

zlib-ng can be compiled to whatever target arch is necessary, and the original post doesn't mention how it was compiled and what architecture and so on.

It's another case not to trust micro benchmarks

For example, he says they didn’t set out to improve the code, but they were porting decennia-old C code to rust. Given the subject (truetype font parsing and rendering), my guess would be that the original code had more memory copies copying data out of the font data because rust makes it easier to safely avoid that (in which case the conclusion would be “C could be as fast, but with a lot more effort”), but it could also be that they spent a day figuring out some code did to realize that it wasn’t necessary on anything after Windows 95, and stripped it out, rather than porting it.

Yes, if your code in Lang-X is faster than C, it's almost certainly a skill issue somewhere in the C implementation.

However, in the day-to-day, if I can make my code run faster in Lang-X than C, especially if I'm using Lang-X for only a couple of months and C potentially for decades, that is absolutely meaningful. Sure, we can make the C code just as fast, but it's not viable to spend that much time and expertise on every small issue.

Outside of "which lang is better" discussions on online forums, it doesn't matter how fast you can theoretically make your program, it matters how fast you actually make it with the constraints your business have (time usually).

I'm always reminded of this video, where the author writes the same program in Rust and Go.

https://www.youtube.com/watch?v=Z0GX2mTUtfo

> Now, the Rust version took me about five times as long as the Go version

> The Go one performed almost identically well

Now this was for netcode rather than number crunching. But I actually had a similar surprise with number crunching, with C# and C++. I wrote the same program (rational approximation of Pi), line for line, in both languages, and the C# version ran faster. Apparently C# aggressively optimizes hot code paths while running, whereas to get that behavior in C++, you need to collect profiler data and use a special compiler flag.

I also wonder how much of an improvement you’d get by just asking for a “simple rewrite” in the existing language. I suspect there are often performance improvements to be had with simple changes in the existing language

As is the case with any languages, of course, it is not in favor (nor against) Rust.

They hadn't expected any perf improvements at all. Quite the opposite, in fact. They were surprised that they saw perf improvements right away.

5-15% is a big deal for a low-level foundational code, especially if you get it along with some other guarantees, which may be of greater importance.

Furthermore, they said that they did not expect any performance gains. They did the rewrite for other reasons and got the unexpected bonus of extra performance.

I'm currently working with ~150 dependencies in my current project which I know would be a major hurdle in previous C or C++ projects.