Show HN:Linux 命令行工具,为长时间运行的 Bash 操作提供互斥锁
Show HN: Linux CLI tool to provide mutex locks for long running bash ops

原始链接: https://github.com/bigattichouse/waitlock

WaitLock 是一个命令行工具,用于在 POSIX 系统的 shell 脚本中管理互斥锁和信号量。 它允许多个进程同步访问资源,提供诸如独占或并发锁持有、进程终止时的自动清理以及 CPU 感知锁定等功能。 主要功能包括:获取和释放锁,在持有锁的情况下执行命令,以各种格式(人类可读、CSV、空字符分隔)列出活动锁(包括陈旧锁),以及检查锁的可用性。 WaitLock 通过环境变量、syslog 和标准输入与 UNIX 环境良好集成。 它支持可自定义的超时时间、调试选项,并且可以配置为使用特定的锁目录。 该工具的 C 实现确保了其可移植性,可在 Linux、macOS 和 BSD 变体上运行。 WaitLock 提供强大的错误处理和资源池管理,支持从基本的独占访问到使用 NFS 的分布式锁定等场景。 安装过程简单直接,可以通过包管理器或从源代码构建。

一个新开发的 Linux 命令行工具,借助 Claude (一种人工智能工具) 创建,旨在简化长时间运行的 bash 操作中的互斥锁/信号量锁定。该工具提供单个命令 `waitlock` 来管理独占执行,具有列出当前锁和允许多个进程持有锁等功能。 Hacker News 上的讨论指出,已经存在像 `flock` 这样的标准 Unix 工具,但这个新工具提供了更好的易用性——特别是等待和释放锁的语法更简单,以及额外的便利功能。一些评论员认为,这个问题最好用完整的编程语言来解决,而不是 bash 脚本,而另一些人则承认它在特定场景下很有用,例如管理可能重叠的 cron 任务。开发者构建这个工具既是为了与 Claude 一起学习,也是为了满足个人对更好 cron 任务管理的需求。 该项目在 GitHub 上可用:[https://github.com/bigattichouse](https://github.com/bigattichouse)。
相关文章

原文

WaitLock is a portable UNIX/POSIX command-line tool that provides mutex and semaphore functionality for shell scripts. It enables synchronized access to resources across multiple processes with automatic cleanup when processes die.

Build Status License Version

  • Mutex Mode: Single lock holder (default)
  • Semaphore Mode: Multiple concurrent lock holders
  • Automatic Cleanup: Locks released when process dies
  • CPU-aware Locking: Can scale locks to CPU count
  • Lock Inspection: List and check active locks
  • Multiple Output Formats: Human, CSV, and null-separated
  • Command Execution: Run commands while holding locks
  • UNIX Integration: Environment variables, stdin, syslog
  • Portable C Implementation: Runs on any POSIX system
# Install dependencies (Ubuntu/Debian)
sudo apt-get install build-essential autoconf

# Build and install
./configure
make
sudo make install

# Basic usage - acquire exclusive lock
waitlock myapp &
# ... do exclusive work ...
kill $!

# Execute command with lock
waitlock database_backup --exec "/usr/local/bin/backup.sh --daily"

# List active locks
waitlock --list
  • C compiler (gcc, clang, or compatible)
  • GNU Make
  • autoconf (for building from git)
# Clone the repository
git clone https://github.com/user/waitlock.git
cd waitlock

# Generate configure script (if building from git)
autoreconf -fi

# Configure and build
./configure
make

# Run tests
make check

# Install system-wide
sudo make install

# Or install to custom prefix
./configure --prefix=/usr/local
make install
# Debug build
./configure CFLAGS="-g -O0 -DDEBUG"

# Release build with optimizations
./configure CFLAGS="-O2 -DNDEBUG"

# Cross-compilation example
./configure --host=arm-linux-gnueabihf
# Ubuntu/Debian (when available)
sudo apt-get install waitlock

# CentOS/RHEL (when available)
sudo yum install waitlock

# macOS with Homebrew (when available)
brew install waitlock
waitlock [options] <descriptor>
waitlock --list [--format=<fmt>] [--all|--stale-only]
waitlock --check <descriptor>
echo <descriptor> | waitlock [options]
# Acquire mutex lock
waitlock myapp &
JOB_PID=$!
# ... do exclusive work ...
kill $JOB_PID

# Check if lock is available
if waitlock --check myapp; then
    echo "Lock is available"
else
    echo "Lock is held by another process"
fi

# Execute command while holding lock
waitlock backup_job --exec rsync -av /source /destination

# Use with timeout
waitlock --timeout 30 critical_resource || echo "Timeout!"

1. Basic Mutex (Exclusive Access)

#!/bin/bash
# Ensure only one backup process runs at a time

waitlock database_backup || {
    echo "Another backup is already running"
    exit 1
}

# Perform backup
mysqldump --all-databases > backup.sql
gzip backup.sql

# Lock automatically released when script exits

2. Semaphore (Multiple Concurrent Access)

#!/bin/bash
# Allow up to 4 concurrent download processes

waitlock --allowMultiple 4 download_pool || {
    echo "Too many downloads already running"
    exit 1
}

# Perform download
wget "https://example.com/file.tar.gz"

# Lock automatically released when script exits
#!/bin/bash
# Use one lock per CPU core, reserving 2 cores for system

waitlock --onePerCPU --excludeCPUs 2 cpu_intensive_task || {
    echo "All CPU slots are busy"
    exit 1
}

# Run CPU-intensive task
./compute_job.sh

4. Command Execution Mode

#!/bin/bash
# Execute command while holding lock (recommended approach)

waitlock database_backup --exec bash -c "
    mysqldump --all-databases > backup.sql
    gzip backup.sql
    echo 'Backup completed'
"

5. Lock Monitoring and Management

#!/bin/bash
# Monitor active locks

# List all locks in human-readable format
waitlock --list

# List in CSV format for parsing
waitlock --list --format csv

# Show only stale locks
waitlock --list --stale-only

# Count active locks
waitlock --list --format csv | tail -n +2 | wc -l

6. Pipeline and Batch Processing

#!/bin/bash
# Process files with controlled parallelism

find /data -name "*.csv" | while read file; do
    basename "$file" | waitlock --allowMultiple 3 --exec process_file "$file"
done

# Or with xargs for better performance
find /data -name "*.csv" | \
    xargs -P 10 -I {} sh -c 'waitlock -m 3 batch_processor --exec "process_file {}"'

7. Using with Environment Variables

#!/bin/bash
# Configure via environment variables

export WAITLOCK_TIMEOUT=60
export WAITLOCK_DIR="/var/lock/myapp"
export WAITLOCK_DEBUG=1

waitlock myapp_task --syslog --syslog-facility local0
#!/bin/bash
# Robust error handling

waitlock --timeout 30 critical_resource
case $? in
    0) echo "Lock acquired successfully" ;;
    1) echo "Lock is busy" >&2; exit 1 ;;
    2) echo "Timeout expired" >&2; exit 1 ;;
    3) echo "Usage error" >&2; exit 1 ;;
    *) echo "Unexpected error" >&2; exit 1 ;;
esac

# Your critical section here
perform_critical_operation

9. Resource Pool Management

#!/bin/bash
# Manage GPU resources

# Export slot number for GPU selection
waitlock --allowMultiple 4 gpu_pool &
LOCK_PID=$!

# Wait for lock and get slot number
wait $LOCK_PID
if [ $? -eq 0 ]; then
    # Use WAITLOCK_SLOT environment variable
    export CUDA_VISIBLE_DEVICES=$WAITLOCK_SLOT
    ./gpu_computation.py
fi

10. Distributed Locking (NFS)

#!/bin/bash
# Coordinate across multiple machines using NFS

export WAITLOCK_DIR="/mnt/shared/locks"

waitlock cluster_job --timeout 300 --exec bash -c "
    echo 'Running on $(hostname)'
    ./distributed_task.sh
"
Option Description
-m, --allowMultiple N Allow N concurrent holders (semaphore mode)
-c, --onePerCPU Allow one lock per CPU core
-x, --excludeCPUs N Reserve N CPUs (reduce available locks by N)
-t, --timeout SECS Maximum wait time before giving up
--check Test if lock is available without acquiring
-e, --exec CMD Execute command while holding lock
Option Description
-q, --quiet Suppress all non-error output
-v, --verbose Verbose output for debugging
-f, --format FMT Output format: human, csv, null
--syslog Log operations to syslog
--syslog-facility FAC Syslog facility (daemon|local0-7)
Option Description
-l, --list List active locks and exit
-a, --all Include stale locks in list
--stale-only Show only stale locks
Option Description
-d, --lock-dir DIR Directory for lock files
-h, --help Show usage information
-V, --version Show version information
Variable Description Default
WAITLOCK_DIR Lock directory path auto-detect
WAITLOCK_TIMEOUT Default timeout in seconds infinite
WAITLOCK_DEBUG Enable debug output disabled
WAITLOCK_SLOT Preferred semaphore slot auto

Environment Variable Examples

# Set default timeout
export WAITLOCK_TIMEOUT=300

# Use custom lock directory
export WAITLOCK_DIR="/var/lock/myapp"

# Enable debug output
export WAITLOCK_DEBUG=1

# Prefer specific semaphore slot
export WAITLOCK_SLOT=2
Code Meaning
0 Success
1 Lock is busy
2 Timeout expired
3 Usage error
4 System error
5 Permission denied
6 Lock directory not accessible
75 Temporary failure
126 Command not executable
127 Command not found
# Log all operations to syslog
waitlock --syslog --syslog-facility local0 myapp

# Monitor syslog for lock operations
tail -f /var/log/syslog | grep waitlock

WaitLock uses binary lock files with the following structure:

  • Magic number (0x57414C4B = "WALK")
  • Process metadata (PID, PPID, UID)
  • Lock information (type, slot, max holders)
  • Timestamps and command line
  • CRC32 checksum for integrity

WaitLock is tested on:

  • Linux (glibc, musl)
  • FreeBSD
  • OpenBSD
  • NetBSD
  • macOS

Performance Considerations

  • Lock files are stored in /var/lock/waitlock (system) or /tmp/waitlock (user)
  • Directory scanning is O(n) where n = number of lock files
  • Use hierarchical descriptors for namespace separation
  • Consider tmpfs for high-frequency locking
  1. Permission Denied

    # Check directory permissions
    ls -la /var/lock/waitlock
    
    # Use user-specific directory
    export WAITLOCK_DIR="$HOME/.waitlock"
  2. Stale Locks

    # List stale locks
    waitlock --list --stale-only
    
    # Clean up automatically (locks are cleaned on next access)
    waitlock --check any_descriptor
  3. High Contention

    # Monitor lock contention
    waitlock --verbose --timeout 1 busy_resource
    
    # Use exponential backoff (built-in)
    waitlock --timeout 60 busy_resource
# Enable debug output
export WAITLOCK_DEBUG=1
waitlock --verbose myapp

# Or use command line
waitlock --verbose myapp
# Clone repository
git clone https://github.com/user/waitlock.git
cd waitlock

# Install development dependencies
sudo apt-get install autoconf automake libtool

# Generate build files
autoreconf -fi

# Configure for development
./configure --enable-debug CFLAGS="-g -O0"

# Build and test
make
make check
# Run internal test suite
./src/waitlock --test

# Run shell-based tests
./test_basic.sh
./test_semaphore.sh
./test_timeout.sh
  • Follow POSIX C89/C90 standards
  • Use 4-space indentation
  • Include comprehensive error handling
  • Add tests for new features
  1. Fork the repository
  2. Create a feature branch
  3. Make changes with tests
  4. Submit a pull request

WaitLock is released under the MIT License. See LICENSE for details.

WaitLock was designed following UNIX philosophy principles and inspired by tools like flock(1), lockfile(1), and sem(1). Special thanks to the POSIX standards committee for providing a solid foundation for portable system programming.

联系我们 contact @ memedata.com