Why I picked Shrine over ActiveStorage and CarrierWave for direct-to-S3 uploads, image and video processing, and chunked large-file handling on a Rails app at creator-platform scale.
It was a Tuesday at the creator economy platform I worked at. The branded-mobile-apps squad had just merged a feature letting creators upload course videos directly from their phones. Files in the 400 MB to 2 GB range. Within an hour, our Sidekiq dashboard was a wall of red. Active jobs queued past 4,000. Aurora writer p99 climbed from 8 ms to 280 ms. Two PagerDuty pages were on the Slack thread by the time I joined.
The team had reached for ActiveStorage, the default. It works great until it doesn’t.
I’ve shipped file upload pipelines on three different Rails stacks. Older agency client apps still riding CarrierWave. The big creator-platform monolith leaning on ActiveStorage for everything except the heavy stuff. And greenfield projects where I picked Shrine from day one. This is the post I wish I’d had three years ago.
For anything you’d call production at scale, Shrine wins. ActiveStorage is fine for a side project or an internal tool. CarrierWave is what you maintain, not what you start with today.
ActiveStorage proxies bytes through your Rails app by default. The user POSTs to your server, Rails streams it to S3, sends back a signed_id. On a 2 GB video upload, your Puma worker is held for the entire transfer. Direct uploads exist, but the API is opinionated and the metadata story is thin. Want a checksum verified server-side after a direct upload? You’ll write it yourself. MIME type inspection before accepting the blob? Same.
Shrine flips it. Uploads go straight to S3 from the client using a presigned URL. The Rails app never touches the bytes. The uploader is a plain Ruby class you own, with plugins for derivatives, validations, presigned URLs, backgrounding, and tus.io chunked uploads.
CarrierWave I’ll spend less time on. It predates the attachment patterns that became standard. It’s the reason half the legacy agency client apps had upload code spread across model callbacks, custom uploaders, and ImageMagick shell-outs that broke on every Ruby upgrade.
On every upload over a few megabytes, you want the browser or mobile app PUTting directly to S3. Rails issues a presigned URL, the client uploads, then tells Rails where the file landed. Puma workers stay free.
# Gemfile
gem "shrine", "~> 3.5"
gem "aws-sdk-s3", "~> 1.150"
gem "image_processing", "~> 1.12"
gem "marcel", "~> 1.0"# config/initializers/shrine.rb
require "shrine"
require "shrine/storage/s3"
s3_options = {
bucket: ENV.fetch("S3_UPLOADS_BUCKET"),
region: ENV.fetch("AWS_REGION", "us-east-1"),
access_key_id: ENV["AWS_ACCESS_KEY_ID"],
secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"]
}
Shrine.storages = {
cache: Shrine::Storage::S3.new(prefix: "cache", **s3_options),
store: Shrine::Storage::S3.new(prefix: "store", **s3_options)
}
Shrine.plugin :activerecord
Shrine.plugin :backgrounding
Shrine.plugin :presign_endpoint, presign_options: { method: :put }
Shrine.plugin :determine_mime_type, analyzer: :marcel
Shrine.plugin :validation_helpers
Shrine.plugin :derivatives
Shrine::Attacher.promote_block { CourseVideoPromoteJob.perform_async(record.class.name, record.id, name.to_s, file_data) }
Shrine::Attacher.destroy_block { CourseVideoDestroyJob.perform_async(data) }The presign endpoint lives at one route. Client asks for a URL, PUTs bytes to S3, submits the form with the cache key. The uploader handles the rest.
# app/uploaders/course_video_uploader.rb
class CourseVideoUploader < Shrine
MAX_BYTES = 5.gigabytes
plugin :remove_attachment
plugin :pretty_location, namespace: :record_class
Attacher.validate do
validate_max_size MAX_BYTES, message: "is too large (max 5 GB)"
validate_mime_type %w[video/mp4 video/quicktime video/x-matroska]
validate_extension %w[mp4 mov mkv]
end
Attacher.derivatives do |original|
{
poster: ImageProcessing::Vips
.source(original)
.resize_to_limit(1280, 720)
.convert("jpg")
.call
}
end
endThat validation runs after the cached upload, before promotion to permanent storage. A 6 GB file never gets copied to the store bucket. Cache buckets get a one-day lifecycle rule, the bad file expires, S3 cleans it up.
For images, image_processing on libvips is my default. ImageMagick works. Vips is faster and uses a fraction of the memory. On a Sidekiq worker chewing through hundreds of profile photo derivatives per minute, the memory difference matters.
# app/uploaders/profile_image_uploader.rb
class ProfileImageUploader < Shrine
Attacher.derivatives do |original|
vips = ImageProcessing::Vips.source(original)
{
thumb: vips.resize_to_limit(150, 150).convert("webp").call,
medium: vips.resize_to_limit(640, 640).convert("webp").call,
large: vips.resize_to_limit(1280, 1280).convert("webp").call
}
end
endFor video, never let Sidekiq workers run ffmpeg. They’ll sit on CPU for ten minutes and starve every other job. Offload to MediaConvert or Mux. The Rails side just orchestrates.
# app/jobs/course_video_promote_job.rb
class CourseVideoPromoteJob
include Sidekiq::Job
sidekiq_options queue: :uploads_promote, retry: 5
def perform(record_class, record_id, attachment, file_data)
attacher = Shrine::Attacher.retrieve(model: record_class.constantize.find(record_id), name: attachment, file: file_data)
attacher.atomic_promote
MediaConvertSubmitJob.perform_async(record_id, attacher.url)
rescue Shrine::AttachmentChanged, ActiveRecord::RecordNotFound
# caller already replaced the file or destroyed the record; safe to drop
end
endMediaConvert handles the encode, posts back to SNS, an SQS consumer updates the row. Rails never burns a CPU minute on transcoding.
For anything past a gigabyte, you want resumable, chunked uploads. tus.io is the protocol I’d pick. Shrine has a first-class plugin for it. tus_ruby_server runs as a Rack app you mount inside Rails, and tus-js-client on the browser side handles retries, resumes, and chunk reassembly.
# config.ru
require "tus/server"
map "/files" do
use Rack::Auth::Basic do |username, password|
ActiveSupport::SecurityUtils.secure_compare(username, ENV["TUS_USER"]) &
ActiveSupport::SecurityUtils.secure_compare(password, ENV["TUS_PASSWORD"])
end
Tus::Server.storage = Tus::Storage::S3.new(bucket: ENV.fetch("S3_TUS_BUCKET"))
run Tus::Server
end
map "/" do
run Rails.application
endThe client uploads in 5 MB chunks. If the user’s phone drops off LTE mid-upload, the next attempt resumes from the last acknowledged byte. ActiveStorage has nothing equivalent. You can hack DirectUpload to do multipart chunking, but then you’re writing the resume logic, integrity checks, and cleanup yourself.
Second story, related shape, different lesson.
We had a worker reacting to S3 ObjectCreated events to mark attachments scanned-and-promoted. The handler returned 200 OK to the SNS notification, marked the row, moved on. One Wednesday a creator opened a ticket: “Every video I upload shows up three times in my course.”
Pulled the logs. SNS had retried delivery on a handful of notifications where our endpoint returned 200 slightly past its deadline. No idempotency check. Every retry created a new row. A few hundred customer assets across dozens of branded apps got tangled up.
The first fix that went out was visible-only. We hid the duplicate rows on the frontend. The customer escalated within a day, because Apple had already billed end users for courses now stuffed with duplicate lessons. Hiding the symptom doesn’t refund anything.
Real fix: rewrote the handler with a Sidekiq job plus a database-level unique constraint on (s3_bucket, s3_key, event_id). The endpoint returns 200 within 5 seconds by enqueueing work, the worker dedupes on insert. SNS retries became idempotent at the queue level.
# app/jobs/s3_object_created_job.rb
class S3ObjectCreatedJob
include Sidekiq::Job
sidekiq_options queue: :s3_events, retry: 10
def perform(payload)
record = payload.fetch("Records").first
bucket = record.dig("s3", "bucket", "name")
key = record.dig("s3", "object", "key")
event_id = record.fetch("eventTime") + ":" + record.fetch("requestParameters", {}).fetch("sourceIPAddress", "")
UploadEvent.create!(s3_bucket: bucket, s3_key: key, event_id: event_id)
UploadPromoter.new(bucket: bucket, key: key).call
rescue ActiveRecord::RecordNotUnique
# duplicate SNS delivery, already handled
end
endServer-to-server notifications retry. All of them. Idempotency keys aren’t optional, they’re the contract. I learned that one twice, once with Apple’s SubscriptionRenewal webhooks, once with S3 events. Now I write the dedupe constraint before the handler.
For a new Rails app today, Shrine. Direct uploads to S3 via presigned URLs, image_processing on libvips for derivatives, MediaConvert for video, tus.io for files past a gigabyte. ActiveStorage if you’re shipping an internal tool with a few hundred uploads a day. CarrierWave only if you’re maintaining an app that already has it, and even then I’d plan the migration.
The places ActiveStorage falls down are the places production lives. Direct upload customization, MIME validation before bytes hit your store bucket, backgrounded promotion that survives a worker death, chunked uploads, multi-step derivatives. You can bolt all of that on. Or pick the library that ships with all of it and a plugin system that doesn’t fight you.
Thanks for reading. If you’ve got thoughts, send them my way.