当你还在通过SSH连接时删除了Linux系统中的/lib目录。
When you deleted /lib on Linux while still connected via SSH (2022)

原始链接: https://tinyhack.com/2022/09/16/when-you-deleted-lib-on-linux-while-still-connected-via-ssh/

删除 Linux 系统中的 `/lib` 目录会破坏关键的动态链接,导致大多数命令无法使用,包括 SSH 和 `tmux`。理想的解决方案是替换丢失的文件,但这在没有 `ld-linux` 的情况下非常困难。安装静态版本的 `busybox` 是最佳的救援方案,它允许你使用 `wget` 下载替换库。 如果不存在静态二进制文件,bash 的内置函数可以下载一个静态的 `busybox`。你可以使用诸如 printf 等内置函数。 对于只有像 `dash` 这样基本 shell 的系统,你可以使用 `printf` 来重建一个小的静态二进制文件,编译一个“文件描述符 I/O”(`fdio`)程序并将其转换为八进制表示以便 `printf` 使用。然后可以使用这个 `fdio` 程序通过 shell 重定向,通过 TCP 接收一个静态的 `busybox`。用 `fdio` 覆盖现有的二进制文件,然后用它接收 `busybox`,从而能够进一步恢复系统。

一篇Hacker News帖子讨论了通过SSH连接时意外删除Linux系统`/lib`目录的后果。评论者`todsacerdoti`最初发布了`tinyhack.com`的链接。`jmclnx`指出,较旧的Linux发行版过去会在`/bin`或`/sbin`中包含静态链接的`bash`以减轻此类问题,但现在似乎不再如此,这可能是由于`/bin`和`/usr/bin`合并造成的。另一位评论者`smw`讲述了在生产Solaris机器上发生的类似事件,有人移动了`/lib`而不是删除它。幸运的是,在Solaris上,`/sbin`中的一些二进制文件,包括`ln`,是静态链接的,允许它们使用硬链接来恢复`/lib`并恢复系统。
相关文章
  • (评论) 2024-06-11
  • Linux 危机工具 2024-03-25
  • XZ后门故事——初步分析 2024-04-14
  • (评论) 2024-03-31
  • (评论) 2023-11-17

  • 原文

    Let’s first not talk about why this can happen, but deleting /lib, /usr/lib, or some other essential runtime files happens quite a lot (as you can see: here, here, here, and here). In this post, I will only discuss what happens when you delete /lib on Linux and how to recover from that.

    The easy solution for everything is to replace the missing files, but this can be difficult if /lib is deleted because we won’t have ld-linux, which is needed to run any dynamic executable. When you deleted /lib, all non-static executable (such as ls, cat, etc, will output):

    No such file or directory
    

    You will also be unable to open any new connection using ssh, or open a new tmux window/pane if you are using tmux. So you can only rely on your current shell built in, and some static executables that you have on the system.

    If you have a static busybox installed, then it can be your rescue. You can use wget from busybox to download libraries from a clean system. For your information: Debian has busybox installed by default, but the default is not the static version.

    Minimal Debian install

    If you are worried that this kind of problem might happen to you in the future: Install the static version of the busybox binary, and confirm that it is the correct version.

    Installing static busybox

    Bash to the rescue

    I assume right now that you don’t have a static busybox, and you don’t even have any static executables (which is the situation in many cases, like in the default install of minimal Debian). My solution for this is to download a static busybox from another machine.

    I also assume that you have bash installed (which is the default for most systems). Bash has a lot of default built-ins that we can use. There is a solution from here that can be used to download a file using only built-in bash functions. Other solutions on this thread rely on external command (such as cat). Please note that you need to set the environment variable LANG to C; Otherwise, this script will incorrectly handle Unicode bytes.

    Of course, we can’t chmod the destination file to be executable, so we need to overwrite an existing executable. If you have busybox installed (even if it is the non-static version), you can overwrite this file. At this point, you can start the rescue mission: for example, use wget to download fresh /lib from another system.

    Please note that busybox can’t function with a name that is not a busybox applet name. So if you overwrite for example, the fmt binary with busybox, then it won’t work (it will say: applet not found). If you don’t have busybox, I suggest overwriting cp, then you can use cp to create a copy of cp as busybox (which will be executable).

    cp to busybox

    No bash? printf can help

    If you have a more advanced shell (e.g: zsh), it has TCP modules already built in. You can easily use nc from another machine to send a file to the target machine. Now, let’s assume that you have a very basic shell, for example: dash. Most shell (including dash), has printf as built-in, and we can use this to construct binary files.

    Most (all?) shell’s built-in printf implementation supports \ooo where ooo is 3 digit octal. First approach is to just convert busybox, but this file is quite big (2 megabyte). Copy-pasting large printf commands is tedious and is error-prone. We need a small static binary that can help us.

    This printf trick will also work for other OS, if you can create a small binary for that OS.

    Creating a small ELF for Linux

    You can create a very tiny executable if you use assembly directly, but let’s try to do this using C, so it can be portable across different architectures. The smallest useful program that I can think of is just to copy from stdin to stdout, so we can prepare netcat on a machine:

    cat busybox | nc -v -l -p 10000

    and then we can do this from the borked machine:

    fdio < /dev/tcp/192.168.1.168/10000 > busybox

    The source code can be like this:

    #include "unistd.h"
    
    int main()
    {
            char x;
            while (1) {
                    int c = read(0, &x, 1);
                    if (c!=0) break;
                    c = write(1, &x, 1);
                    if (c!=0) break;
            }
            return 0;
    }
    

    If we try to compile this with standard C library (on AMD64 machine), the result is 776KB.

    $ gcc -Os -static fd.c
    $ du -hs a.out
    768K    a.out

    The Linux kernel source code contains a nolibc implementation that we can use. Using this compilation option:

    gcc -Os -Wl,--build-id=none -fno-asynchronous-unwind-tables -fno-ident -s -nostdlib -nodefaultlibs -static -include nolibc.h fd.c -lgcc -o fd

    We get a 4536 bytes binary. Quite good. If we add -z max-page-size=0x04, we can even get a smaller size.

    gcc -Os -Wl,--build-id=none -z max-page-size=0x04 -fno-asynchronous-unwind-tables -fno-ident -s -nostdlib -nodefaultlibs -static -include nolibc.h fd.c -lgcc -o fd

    It is now 672 bytes. Small enough to transfer. We can convert this using Python.

    import sys
    
    with open(sys.argv[1], "rb") as f:
        data = f.read()
    
    start = 0
    width = 20
    targetname = sys.argv[2]
    while True:
        part = data[start:start+width]
        if part=='':
            break
        a = ''.join(['\\'+(oct(ord(i)).zfill(3))[-3:] for i in part])
        dest = '>'
        if start>0:
            dest += '>'
        dest += ' ' + targetname
        print("printf '{}' {} ".format(a, dest))
        start += width
    
    

    We can then copy paste this to our ssh session, then do the /dev/tcp redirection trick.

    Output example

    Of course, we can also write a complete program that makes the TCP connection instead of relying on bash redirection.

    I hope you will never need this knowledge

    This problem occurred to me a few days ago when I updated my Solar Powered Pi Zero, and somehow /lib got deleted (not sure what caused it). This is not a very important machine, and I could have just reimaged the MicroSD card and be done with it, but I was curious if I could recover from the error.

    I hope you will never have this error on your production/important machine, but if you have this problem in the future, I hope this post will help you recover from the situation.

    联系我们 contact @ memedata.com