Athena is a lightweight Change Data Capture (CDC) solution that streams changes from Microsoft SQL Server to Apache Kafka. Built in Golang, it supports SASL-authenticated Kafka brokers and provides a straightforward setup experience. Unlike alternatives such as Debezium, which can be complex to configure and manage. Athena offers greater simplicity and operational ease. It automatically manages CDC setups, publishes database changes to a single Kafka topic, and delivers a clean, intuitive event format that is easy for downstream consumers to understand and process.

• Creates a message for changes like create, update, delete for rows in MSSQL database tables to a single Kafka topic.
• Athena only creates messages for all new table changes. Existing ones are ignored.
• Kafka topic have to be created before hand. Unlike Debezium, Athena will not create the topic own it's own.
• All CDC setups in MSSQL is automatically done by Athena when setup command is run.
• By default, Athena will poll for changes for all tables, you can use the skippedTables option in the config.json to ignore any tables.

You can download the pre-compiled binaries from the Github releases page and copy them to the desired location. After that you can follow the below steps in order.

You can find more details about the paramters in config file in below sections.

{
    "dbHost": "127.0.0.1",
    "dbPort": 1433,
    "dbUser": "",
    "dbPassword": "",
    "dbName": "",

    "kafkaHost": "",
    "kafkaEnableTLS": false,
    "kafkaTopic": "",

    "kafkaSASLMechanisms": "NONE",
    "kafkaSASLUsername": "",
    "kafkaSASLPassword": "",

    "pollInterval": 10,
    "fetchLimit": 50,
    "skippedTables": [],

    // If you want to collect logs in clickhouse
    "clickHouse": false,
    "clickHouseHost": "<host>:<port>",
    "clickHouseUsername": "",
    "clickHousePassword": "",
    "clickHouseDatabase": "",
    "clickHouseTableName": "",
    "clickHouseTableTTL": 12
}

cd /etc/systemd/system
nano athena_mssql_kafka.service

[Unit]
Description=Athena MSSQL Kafka Service
After=network.target

[Service]
Type=simple
ExecStart=athena run

[Install]
WantedBy=multi-user.target

Please note that path in ExecStart needs to change while creating the service file.

systemctl start athena_mmsql_kafka.service
systemctl status athena_mmsql_kafka.service

Athena can be configured using the config.json file created on the root the Athena binary. Here are the details of the configuration keys and what they do in table format.

Athena executable have some other helper functions apart from setup or run which are explained below. These can be run like eg: ./athena uninstall

For setting up development environment, there is a docker file in the folder dev. It will create all necessary services like MSSQL with sample database, Kafka etc. This same environment can be used for running integration tests.

• Install latest version of Go from here.
• Clone that project from Github.
• Run go mod download command to install all mods.
• Then run the commands below as needed.

cd dev
docker composer up -d
cd ..
set GORUN=true # Used for identifying if script is run from go run command to choose correct path for reading config.json or db.sqlite
go run . setup
go run . run

Please note that the docker-compose.yml in the dev folder should only be used for development purpose.

Before running the tests, make sure you have setup the development environment and also config.json is setup correctly.

cd tests
go test -v -run TestIntegration

You can build the binaries or do development of Athena by following the below steps. Athena is build fully on Golang. So you should install latest version of Go from here. Do note that building binaries are managed with the Goreleaser.

• Clone that project from Github.
• Run go mod download command to install all mods.
• Run SET GORUN=true command to set gorun variable.
• Run the command goreleaser release --snapshot --clean for building the binaries.

Athena is licensed under the MIT License.