[Rails] Active Storage how to validate file type

rails_active_storage_how_to_validate_file_type.md

Rails Active Storage how to restrict uploadable file types

Active Storage doesn't have validations yet.

We can restrict the accepted file types in the form:

<div class="field">
  <%= f.label :deliverable %>
  <%= f.file_field :deliverable, direct_upload: true, 
    accept: 'application/pdf, 
    application/zip,application/vnd.openxmlformats-officedocument.wordprocessingml.document' %>
 </div>

And add a custom validation in the model:

class Item
  has_one_attached :document

  validate :correct_document_mime_type

  private

  def correct_document_mime_type
    if document.attached? && !document.content_type.in?(%w(application/msword application/pdf))
      errors.add(:document, 'Must be a PDF or a DOC file')
    end
  end
end

Source: https://stackoverflow.com/questions/48349072/ruby-on-rails-active-storage-how-to-accept-only-pdf-and-doc?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa

This is useful, but has one caveat that one might or might not care about, so I'm writing it out here in case it helps anyone.

The validation proposed in this gist will check the file after it is uploaded. This might be a problem especially when using a paid cloud service.

The following is another method, that doesn't send the file over to the cloud service at all. I don't really like it, and it probably breaks the MVC, but I'm not sure there is another way if we don't want the file to be sent.

So what I've done in my case, is validate in the controller. Yes, it's weird. But even calling object.file = will upload our file to ie S3 even if we don't call object.save!

We can make it a bit nicer by having a constant in the model like this:

  FILE_VALIDATIONS = {
    content_types: ["application/pdf", "image/jpeg", "image/png"],
    max_size: 5.megabytes,
    min_size: 1.kilobyte
  }

The controller can use this contant then and check with the param[:file].content_type and param[:file].size to check if the file can be accepted. The controller can then add errors on the object without assigning anything to the file field.

@b-nik 👍 this is a solid point. I came across dozens of replies suggesting validating with document.attached?

Try out this gem: igorkasyanchuk/active_storage_validations

What it can do

• validates if file(s) attached
• validates content type
• validates size of files
• validates number of uploaded files (min/max required)
• validates dimension (pull request)
• custom error messages

Is there any way to validate file upload content type, before upload, when using direct_upload: true ?

@artur79 I'd imagine not since you're bypassing your server and uploading directly from a user to your storage medium. The only place capable of doing validation in that scenario is in JS on the upload side, and that's generally a bad idea unless you trust your users because they could disable or tamper with the JS validation.

@drusepth makes sense :)

Here's a tough one I've had a client able to bypass content type security like the above with. Basically, they were able to upload a .png file with non-png file content:

Content-Disposition: form-data; name="attachment[file]"; filename="eicar.png"
Content-Type: application/octet-stream
X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*

I don't think content type(mime type) checking is good enough to securely validate files. @phlegx does that active_storage_validations gem do this level of content checking? Same question to @ConfusedVorlon with the gem they linked.

If not, does anyone have an idea of how to implement such a content check?

I came here with the same question as @shawndeprey: I don't want to use the client-supplied content_type but let the server determine it by inspecting the actual file contents.

Anyone who does this already?

You'll have to use some other approach to validate whether the attachment is what it claims to be.

If you're working with images, then imagemagick is probably an obvious choice.

the way I'd do it is

1. have a 'validated' column in my model (default false)
2. kick off a background job on create (or if the image changes)
3. use imageMagick to validate the image
4. update validated=true if it passes, delete if not

@ConfusedVorlon thank you for the suggestion! I'm a bit confused that secure content type validation is not part of ActiveStorage though. We just had a big Rails app pentested and (apart from a handful of hosting configurations), the ActiveStorage type insecurity was the ONLY thing the security company found.

Edit: actually, it seems it is secure. After create and committing the ActiveStorage::Attachment, it calls the identify() method on the Blob, which determines the Mime type by itself, overwriting the previously supplied one and setting the field "identified" to true in the meta hash of the Blob. It seems we're making some kind of mistake here... Have to dig deeper.

Storing stuff is a job for your suitcase.
Checking that there isn't a bomb in there is a job for the xray, the mass spectrometer and security team at the airport.

They're just completely separate jobs.

Add to that the fact that you could be storing images, word documents, music files, videos, gifs, pdfs, .mmw files (a custom file format for one of my apps) or a gazillion other things. It would be ridiculous for rails to try to build secure validation for all those types into their 'suitcase' functionality.

Yes, but sticking to your example, the security team checks all the deposited suitcases before storing them (in the planes). The airport needs to handle both checking and storage.

In reality, Rails is a secure web framework, so I was confused it simply seemed not to offer the xray. It does though, sometimes we seem to cause "empty" uploads which cannot be analyzed (because they're empty), so that's a fault on my site. Rails' xray is in place and works. :)

undefined method `content_type' for #ActiveStorage::Attached::Many:0x00007fdb4cbce6f0
How to resolve this?

@b-nik, Rails 6 has fixed the issue (Store newly-uploaded files on save rather than assignment). rails/rails@e8682c5

Of the two available gems I went with https://github.com/aki77/activestorage-validator as it is much simpler. You probably don't even need the gem as you could just bring in the validator class.