Local Kinesis in a few MB.

A small Amazon Kinesis Data Streams emulator written in Rust.

Configure your AWS SDK to use http://localhost:4567 with dummy credentials.

It implements the ten Kinesis Data Streams operations that PutRecord producers and polling consumers use.

A single static binary with no JVM, Node.js or Python runtime underneath: it starts in milliseconds and idles at a few MB of RAM.

$ docker run --rm -p 4567:4567 ghcr.io/pafin-inc/fakestream
Get it on GitHub → Read the docs
$ ./fakestream
INFO fakestream listening (Kinesis, AWS JSON 1.1) address=0.0.0.0:4567
INFO default retention retention_secs=86400
INFO persistence disabled

The problem

Running all of LocalStack for one Kinesis stream is expensive.

LocalStack serves Kinesis from inside the full LocalStack container: a Python control plane plus a Node.js engine, or a JVM if you enable the Scala engine for throughput. In our local stack it used several hundred MB before the first record arrived, and the JVM engine wanted a multi-GB heap.

As of March 2026 LocalStack also requires an account and an auth token, and its free tier is limited to non-commercial use.

fakestream was written for teams whose applications only need the Kinesis polling APIs.

Faster dev loop

Starts in milliseconds, so restarting it between test runs or on every file change is cheap.

Cheaper, faster CI

Each CI job can run its own instance. Memory is a few MB idle and grows with the records it retains.

One process

A single process serving one API, with no control plane or other service emulators alongside it.

Production unchanged

Set the endpoint override in local and CI configuration only. Production configuration is not touched.

Keep LocalStack or real AWS for everything else. Current AWS SDKs and the CLI read AWS_ENDPOINT_URL_KINESIS, so only Kinesis traffic is redirected; older SDKs take the endpoint on the Kinesis client instead.

By the numbers

Footprint and throughput from one benchmark.

~2 MB
Idle RSS, release build, before any stream exists
171 MB/s
PutRecord: 490 records/s of ≈350 KB, 8 concurrent producers
227 MB/s
GetRecords: 619 records/s of ≈370 KB served to one consumer

Measured on a developer laptop with a replay of pafin's own ingestion traffic (large batched records), at about 12× that pipeline's peak rate; the load generator never saturated the server. Numbers will differ on your hardware.

Features

What is implemented.

Works with the standard AWS SDKs

Any client that speaks AWS JSON 1.1 works with an endpoint override and dummy credentials; Java SDKs need CBOR turned off. Implemented: CreateStream, DeleteStream, PutRecord, PutRecords, GetRecords, GetShardIterator (all five iterator types), DescribeStream, DescribeStreamSummary, ListShards and ListStreams.

Tiny footprint

A small fixed thread pool and one RwLock<Store>, no async runtime. A few MB idle; beyond that, memory tracks the bytes held in streams plus per-record overhead, and shrinks as records expire.

Optional persistence

--persist <dir> turns on a segmented write-ahead log, so records survive a restart. A hard crash can lose records written since the last maintenance tick (5 s by default). In our benchmark it cost about 8% of PutRecord throughput.

Kinesis limits enforced

1 MiB record and 5 MiB batch limits, 5-minute iterator expiry, 10 MiB GetRecords responses. Requests that exceed them fail in local tests rather than in production. Per-shard throughput limits are not enforced.

Configurable TTL & retention

--retention-hours sets the default retention for streams created without one. --ttl-seconds overrides it precisely, and 0 keeps records forever.

Single static binary

A musl-linked Rust executable with no JVM, Node.js or Python runtime to install, also published as a Docker image for linux/amd64 and linux/arm64. Configuration is a handful of flags or environment variables.

How it works

One HTTP endpoint, one process.

Two AWS SDK producers send records to fakestream, which hashes each partition key with MD5 and routes each record to one of three shards by hash-key range.
Single endpoint

POST / with X-Amz-Target: Kinesis_20131202.<Op> and application/x-amz-json-1.1 — the same request the AWS SDK sends to the real service, handled directly.

MD5 hash-key ranges

Each partition key is hashed to a 128-bit MD5 value and the record goes to the shard whose contiguous hash-key range contains it, as in Kinesis. A single global counter assigns increasing sequence numbers.

Checkpoints are client-side

fakestream issues shard iterators; durable checkpoints stay in your consumer (DynamoDB with KCL-style pollers, in-process in tests), as with real Kinesis. KCL lease coordination is not implemented.

Quickstart

Run it, point the SDK at it, create a stream.

Run fakestream
$ docker run --rm -p 4567:4567 ghcr.io/pafin-inc/fakestream
Point your AWS SDK at it
export AWS_ENDPOINT_URL_KINESIS=http://localhost:4567
export AWS_ACCESS_KEY_ID=dummy
export AWS_SECRET_ACCESS_KEY=dummy
export AWS_DEFAULT_REGION=us-east-1
Create a stream
$ aws --endpoint-url=http://localhost:4567 kinesis create-stream --stream-name my-stream --shard-count 1
Run your producers and consumers

PutRecord, PutRecords, GetShardIterator and GetRecords behave as they do against Kinesis. Java clients need the CBOR setting from the FAQ.

FAQ

Common questions.

Which APIs are supported?
  • CreateStream, DeleteStream
  • ListStreams, DescribeStream, DescribeStreamSummary, ListShards
  • PutRecord, PutRecords
  • GetShardIterator with TRIM_HORIZON, LATEST, AT_SEQUENCE_NUMBER, AFTER_SEQUENCE_NUMBER, AT_TIMESTAMP
  • GetRecords

The tested path is PutRecord or PutRecords, then GetShardIterator and repeated GetRecords.

Does it persist data across restarts?

Using --persist <dir> enables a segmented write-ahead log (WAL) that survives process restarts.

Without the flag, records exist only in process memory and are discarded when the process exits.

In our benchmark, persistence cost about 8% of PutRecord throughput.

How is it so small?
  • Rust, compiled for size with LTO.
  • A minimal HTTP layer; blocking I/O on a small thread pool instead of an async runtime.
  • No JVM, Node.js or Python interpreter underneath.
Does it replace LocalStack entirely?

fakestream emulates only Kinesis Data Streams. LocalStack can keep serving SQS, DynamoDB, S3 and the rest.

Point only AWS_ENDPOINT_URL_KINESIS at fakestream. Optionally narrow LocalStack with SERVICES=dynamodb,... so it no longer starts its own Kinesis.

Does it work with the Java SDK?

Yes, once CBOR is turned off. The Java SDKs default to application/x-amz-cbor-1.1, which fakestream does not speak.

  • SDK v1: AWS_CBOR_DISABLED=true
  • SDK v2: CBOR_ENABLED=false

Every client still needs the endpoint override. Python, JavaScript, Go and the AWS CLI need nothing beyond that.

What is not supported?
  • Enhanced fan-out (SubscribeToShard)
  • Resharding (SplitShard, MergeShards, UpdateShardCount)
  • Per-shard throughput throttling
  • The CBOR protocol
  • Authentication

fakestream is built for local development and CI. Run it on a trusted network only: localhost, a CI job, or a private container network.

Get the image or the source.

MIT licensed. Images for linux/amd64 and linux/arm64 on GHCR; port 4567; in-memory unless --persist is set.

Get it on GitHub →