0

I'm working on an Rails 4.1 engine that handles user uploads of photos and videos. I'm using Mongoid-Paperclip to handle the uploads and Paperclip-av-transcoder to encode the videos into several formats. All files are stored at S3. All of that works fine, but as you can expect, encoding the videos can take quite some time, so the next step is to make that happen in the background. I did some googling and found Delayed_Paperclip that seems to do what I need. After that it seemed that Sidekiq was the best option for handling the background processing.

Now the problem is, I can't make all this work together. Running my unit tests I get NoMethodError: undefined method 'process_in_background' so it seems the problem resides on Delayed_Paperclip, although there is no special setup to it.

This is the model firing the problem

module MyEngine
  class Video
    include Mongoid::Document
    include Mongoid::Paperclip

    has_mongoid_attached_file :file,
      :path => ':hash.:extension',
      :hash_secret => "the-secret",
      :storage => :s3,
      :url => ':s3_domain_url',
      :s3_credentials => File.join(Rails.root, 'config', 's3.yml'),
      :bucket => "my-bucket-#{Rails.env}",
      :styles => {
        :mp4 => { :format => 'mp4', :convert_options => { :output => { :vcodec => 'libx264', :acodec => 'copy' } } },
        :ogg => { :format => 'ogg', :auto_rotate => true  },
        :webm => { :format => 'webm', :auto_rotate => true  },
        :thumb => { :geometry => "250x187#", :format => 'jpg', :time => 10, :auto_rotate => true }
      },
      :processors => [:transcoder]

    validates_attachment :file, :content_type => { :content_type => ["video/x-flv", "video/mp4", "video/ogg", "video/webm", "video/x-ms-wmv", "video/x-msvideo", "video/quicktime", "video/3gpp"] }

    process_in_background :file
  end
end

I've tried adding require "delayed_paperclip" to the lib/myengine/myengine.rb file but that didn't help.

Regarding Sidekiq, I have added to the test_helper.rb the following:

require 'sidekiq/testing'
Sidekiq::Testing.inline!

Note that I did not forget to run bundle install and Redis is up and running. I'm using Mongoid, not active record.

What I am doing wrong? Has anyone successfully used this setup? Is there another combination of gems that I should try?

Aditional info:

  • Delayed_paperclip 2.9.1
  • Mongoid 4.0.2
  • Mongoid-Paperclip 0.0.9
  • Paperclip 4.2.1
  • Paperclip-av-transcoder 0.6.4
  • Rails 4.1.9
  • Sidekiq 3.5.0
Julien
  • 1,613
  • 15
  • 32

1 Answers1

1

I've been digging through the code of delayed_paperclip and it is definitely tied to ActiveRecord so not compatible with Mongoid. I gave a try to mongoid_paperclip_queue but that gem hasn't been updated in 4 years and doesn't seem to work with the current versions of rails/mongoid/paperclip as far as I can tell.

I therefore decided the best way to solve my issue would be to override the code of delayed_paperclip that integrates with ActiveRecord and make it instead work with Mongoid.

This is what I ended up doing, and seems to be working fine so far:

lib/myengine.rb

require "mongoid_paperclip"
require "paperclip/av/transcoder"
require "delayed_paperclip"
require "myengine/engine"

module Myengine
end

DelayedPaperclip::Railtie.class_eval do

  initializer 'delayed_paperclip.insert_into_mongoid' do |app|
    ActiveSupport.on_load :mongoid do
      DelayedPaperclip::Railtie.insert
    end

    if app.config.respond_to?(:delayed_paperclip_defaults)
      DelayedPaperclip.options.merge!(app.config.delayed_paperclip_defaults)
    end
  end

  # Attachment and URL Generator extends Paperclip
  def self.insert
    Paperclip::Attachment.send(:include, DelayedPaperclip::Attachment)
    Paperclip::UrlGenerator.send(:include, DelayedPaperclip::UrlGenerator)
  end
end

DelayedPaperclip::InstanceMethods.class_eval do

  def enqueue_post_processing_for name
    DelayedPaperclip.enqueue(self.class.name, read_attribute(:id).to_s, name.to_sym)
  end
end

Then all you need is to include the delayed_paperclip glue to the model:

module Myengine
  class Video
    include Mongoid::Document
    include Mongoid::Paperclip
    include DelayedPaperclip::Glue      # <---- Include this

    has_mongoid_attached_file :file,
      :path => ':hash.:extension',
      :hash_secret => "the-secret",
      :storage => :s3,
      :url => ':s3_domain_url',
      :s3_credentials => File.join(Rails.root, 'config', 's3.yml'),
      :bucket => "my-bucket-#{Rails.env}",
      :styles => {
        :mp4 => { :format => 'mp4', :convert_options => { :output => { :vcodec => 'libx264', :acodec => 'copy' } } },
        :ogg => { :format => 'ogg', :auto_rotate => true  },
        :webm => { :format => 'webm', :auto_rotate => true  },
        :thumb => { :geometry => "250x187#", :format => 'jpg', :time => 10, :auto_rotate => true }
      },
      :processors => [:transcoder]

    validates_attachment :file, :content_type => { :content_type => ["video/x-flv", "video/mp4", "video/ogg", "video/webm", "video/x-ms-wmv", "video/x-msvideo", "video/quicktime", "video/3gpp"] }

    process_in_background :file
  end
end

Hopefully this will save somebody else all the trouble.

Julien
  • 1,613
  • 15
  • 32