[{"data":1,"prerenderedAt":44},["Reactive",2],{"featurePages":3,"blogPosts-improving-chatwoot-file-upload":4},[],{"slug":5,"id":6,"uuid":7,"title":8,"html":9,"comment_id":6,"feature_image":10,"featured":11,"visibility":12,"created_at":13,"updated_at":14,"published_at":15,"custom_excerpt":16,"codeinjection_head":17,"codeinjection_foot":17,"custom_template":17,"canonical_url":17,"authors":18,"tags":25,"primary_author":39,"primary_tag":40,"url":41,"excerpt":16,"reading_time":42,"access":43,"comments":11,"og_image":17,"og_title":17,"og_description":17,"twitter_image":17,"twitter_title":17,"twitter_description":17,"meta_title":8,"meta_description":16,"email_subject":17,"frontmatter":17,"feature_image_alt":17,"feature_image_caption":17},"improving-chatwoot-file-upload","628363f74b8f26503f72dede","2e0dc686-fbcc-404f-b1a0-a40a17404d0c","How we improved Chatwoot File Upload","\u003Cp>\u003C/p>\u003Cp>Don't you hate waiting? When you know your internet connection is great, but the site takes a long time to load, or the uploaded files appear broken, or UI rendering is weird – it is frustrating. \u003C/p>\u003Cp>We ended up facing one of the above issues. Actually, not us, but a few of our customers. So yeah, us. :D \u003C/p>\u003Cfigure class=\"kg-card kg-image-card\">\u003Cimg src=\"https://www-internal-blog.chatwoot.com/content/images/2022/05/embarrased.gif\" class=\"kg-image\" alt loading=\"lazy\" width=\"499\" height=\"274\">\u003C/figure>\u003Cp>\u003C/p>\u003Cp>\u003C/p>\u003Ch2 id=\"heres-what-happened\">Here's what happened...\u003C/h2>\u003Cp>\u003C/p>\u003Cp>One of our customers was facing issues with uploading files. So, we dug in. \u003C/p>\u003Cp>One of the major problems in Chatwoot File upload was that the uploaded image appeared broken even if the upload was completed from the front end. \u003C/p>\u003Cp>This happened because uploading the attachment from our application servers to the cloud took time.\u003C/p>\u003Cp>\u003C/p>\u003Ch2 id=\"so-we-talked-about-possible-solutions\">So, we talked about possible solutions\u003C/h2>\u003Cp>\u003C/p>\u003Cp>We considered various methods to improve file upload in Chatwoot:\u003C/p>\u003Cp>\u003Cstrong>Uploading the attachment on the front end and saving it to the cloud from the sidekick\u003C/strong>\u003C/p>\u003Cp>But then delaying the upload would also create a similar issue – we won't be able to fetch/update or delete the message with an attachment unless the upload is completed. So this would bring us back to square one.\u003C/p>\u003Cp>\u003Cstrong>Using \u003Cem>\u003Ccode>aws-sdk-s3\u003C/code>\u003C/em> direct upload, where we send a request to the Amazon S3 cloud with our credentials. \u003C/strong>\u003C/p>\u003Cp>The S3 provides the pre-signed URL to upload the file, the solution which would work because they came with the pre-signed URL just for this problem. \u003C/p>\u003Cp>In one of our previous projects, we had seen a performance improvement with the pre-signed URL issue. Still, we discarded this idea because it is limited to S3 cloud storage, and we can't expect our self-hosted customers to stick to one cloud storage.\u003C/p>\u003Cp>\u003Cstrong>Rails active storage direct upload\u003C/strong>\u003C/p>\u003Cp>We decided to go with this, but wait. \u003C/p>\u003Cp>Rails let you implement this in just three steps, and it works like a charm for all the cloud storage providers supported by Chatwoot.\u003C/p>\u003Col>\u003Cli>Send the request to the \u003Ccode>direct_upload\u003C/code> endpoint provided by rails which is a \u003Ccode>direct_upload_controller\u003C/code>.\u003C/li>\u003Cli>Rails will return the authorized signed key to your cloud storage by reading your \u003Cem>\u003Ccode>storage.yml\u003C/code>.\u003C/em>\u003C/li>\u003Cli>Upload the file directly to the storage with the signed key.\u003C/li>\u003C/ol>\u003Cp>There is no intermediate application server to slow down the upload process.\u003C/p>\u003Cp>Great! This should solve our issue for sure, right?\u003C/p>\u003Cp>But as we progressed with the implementations, we encountered quite a few hurdles. \u003C/p>\u003Cul>\u003Cli>How are we authenticating the user? \u003C/li>\u003Cli>How are we ensuring that the request comes from an authorized user in Chatwoot? \u003C/li>\u003Cli>What if the signed key gets compromised? \u003C/li>\u003Cli>The cloud storage can get piled up, especially if someone bulk uploaded to the storage with many files in the given expiry time.\u003C/li>\u003C/ul>\u003Cp>\u003C/p>\u003Ch2 id=\"we-broke-the-issue-down\">We broke the issue down\u003C/h2>\u003Cp>1.\tWe couldn't use \u003Ccode>rails/active_storage/direct_uploads\u003C/code> endpoint as it doesn't have any authentication check and is extended from the action controller base.\u003C/p>\u003Cp>\u003Cu>Proposed Solution\u003C/u>: We need a separate endpoint for direct upload with the \u003Cem>\u003Ccode>rails/direct_upload\u003C/code>\u003C/em> functionalities.\u003C/p>\u003Cp>2.\tWe should be able to send the auth token with the direct upload signed key request. But it wasn't available with the current DirectUpload JS library and the documentation. \u003C/p>\u003Cp>\u003Cu>Proposed Solution\u003C/u>: We can be more ambitious and build out the extension to the existing library.\u003Cbr>\u003C/p>\u003Ch3 id=\"lets-solve-it-one-by-one\">Let's solve it one by one\u003C/h3>\u003Cp>\u003C/p>\u003Cp>\u003Cstrong>1.\t\u003Cstrong>\u003Cstrong>Add a custom endpoint to get the direct upload signed key\u003C/strong>\u003C/strong>\u003C/strong>\u003C/p>\u003Cp>The first issue and solution were easy to build and could be handled by the current \u003Ccode>direct_upload.js\u003C/code> library.\u003C/p>\u003Cp>We wrote a new controller where we extended \u003Cem>\u003Ccode>ActiveStorage::DirectUploadsController\u003C/code>\u003C/em> and added the logic to check if the user is a valid user asking for the signed key.\u003C/p>\u003Cfigure class=\"kg-card kg-code-card\">\u003Cpre>\u003Ccode class=\"language-ruby\">class Api::V1::Widget::DirectUploadsController < ActiveStorage::DirectUploadsController\n  include WebsiteTokenHelper\n  before_action :set_web_widget\n  before_action :set_contact\n\n  def create\n    return if @contact.nil? || @current_account.nil?\n\n    super\n  end\nend\u003C/code>\u003C/pre>\u003Cfigcaption>Reference: \u003Ca target=\"_blank\" rel=\"noopener\" href=\"https://github.com/chatwoot/chatwoot/blob/62d60f000bda12fb17876fcea39e8527c52feec9/app/controllers/api/v1/widget/direct_uploads_controller.rb?ref=www-internal-blog.chatwoot.com\">direct_uploads_controller.rb\u003C/a>\u003C/figcaption>\u003C/figure>\u003Cpre>\u003Ccode class=\"language-ruby\">class Api::V1::Widget::DirectUploadsController < ActiveStorage::DirectUploadsController\n  include WebsiteTokenHelper\n  before_action :set_web_widget\n  before_action :set_contact\n\n  def create\n    return if @contact.nil? || @current_account.nil?\n\n    super\n  end\nend\u003C/code>\u003C/pre>\u003Cp>We updated the endpoint with the new controller in the new DirectUpload call.\u003C/p>\u003Cpre>\u003Ccode>const upload = new DirectUpload(file, url)\u003C/code>\u003C/pre>\u003Cp>So the first problem was solved. \u003C/p>\u003Cp>We had a new endpoint for direct upload. Hitting the new endpoint from the front end, the controller had a check for the website token and authentication token.\u003C/p>\u003Cp>\u003Cstrong>2.\t\u003Cstrong>\u003Cstrong>Authenticating the direct upload signed key request.\u003C/strong>\u003C/strong>\u003C/strong>\u003C/p>\u003Cp>Now let's move on to the second issue, how could we send the website token and authentication token from the front end to the new endpoint in the new DirectUpload call?\u003C/p>\u003Cp>We checked the \u003Ca target=\"_blank\" rel=\"noopener\" href=\"https://edgeguides.rubyonrails.org/active_storage_overview.html?ref=www-internal-blog.chatwoot.com#direct-uploads\">documentation\u003C/a>. Was there a way to send extra data with the request? We couldn't find any way. \u003C/p>\u003Cp>We checked if there was an event on which we could bind the token, but none was mentioned. There were just file and URL parameters that you could send to DirectUpload class.\u003C/p>\u003Cpre>\u003Ccode class=\"language-javascript\">const url = input.dataset.directUploadUrl\nconst upload = new DirectUpload(file, url)\n\nupload.create((error, blob) => {\n if (error) {\n   // Handle the error\n } else {\n   // Add an appropriately-named hidden input to the form with a\n   //  value of blob.signed_id so that the blob ids will be\n   //  transmitted in the normal upload flow\n   const hiddenField = document.createElement('input')\n   hiddenField.setAttribute('type', 'hidden')\n   hiddenField.setAttribute('value', blob.signed_id)\n   hiddenField.name = input.name\n   document.querySelector('form').appendChild(hiddenField)\n }\n})\u003C/code>\u003C/pre>\u003Cp>\u003Cbr>At one point, we thought about extending the \u003Cem>\u003Ccode>DirectUpload.js\u003C/code>\u003C/em> and making our \u003Cstrong>Chatwoot\u003C/strong> version. But, then we thought: let's open the library first, and hey! So there we had it; there was an event bound to creating a blob process when we sent the request to get the signed key. The event was \u003Cem>\u003Cstrong>\u003Ccode>DirectUploadWillCreateBlobWithXHR\u003C/code>\u003C/strong>\u003C/em>.\u003C/p>\u003Cp>You can call this event while sending the parameters to DirectUpload and add the new tokens in XHR headers.\u003C/p>\u003Cp>Here is how we do it:\u003C/p>\u003Cp>1) We are sending the website token in the URL\u003C/p>\u003Cp>2) Setting the auth token in the XHR header, and\u003C/p>\u003Cp>3) Sending it to the new URL endpoint.\u003C/p>\u003Cpre>\u003Ccode class=\"language-javascript\">const { websiteToken } = window.chatwootWebChannel;\nconst upload = new DirectUpload(\n  file.file,\n  `/api/v1/widget/direct_uploads?website_token=${websiteToken}`,\n  {\n    directUploadWillCreateBlobWithXHR: xhr => {\n      xhr.setRequestHeader('X-Auth-Token', window.authToken);\n    },\n  }\n);\n \nupload.create((error, blob) => {\n  if (error) {\n    window.bus.$emit(BUS_EVENTS.SHOW_ALERT, {\n      message: error,\n    });\n  } else {\n    this.onAttach({\n      file: blob.signed_id,\n      ...this.getLocalFileAttributes(file),\n    });\n  }\n});\u003C/code>\u003C/pre>\u003Cp>I think we met the criteria to resolve the second issue.\u003C/p>\u003Ch2 id=\"to-conclude\">To conclude\u003C/h2>\u003Cp>With both the issues resolved, we were good to go with this feature, which is already a bug fix for one of our customers. :D\u003C/p>\u003Cp>Long story short, rails documentation is not up to the mark, but the feature is, and there is a way to authenticate your direct upload request with this active storage's new feature.\u003C/p>\u003Cp>And you can always refer to our blog if you are stuck at direct upload.\u003C/p>\u003Cp>Here are the links to related GitHub issues:\u003C/p>\u003Cp>https://github.com/chatwoot/chatwoot/issues/3567 https://github.com/chatwoot/chatwoot/issues/4147\u003C/p>\u003Cp>Thanks for reading. ✌️\u003Cbr>\u003C/p>","https://www-internal-blog.chatwoot.com/content/images/2022/07/Chatwoot-engineering.png",false,"public","2022-05-17T08:59:35.000+00:00","2022-07-29T05:09:58.000+00:00","2022-07-28T16:32:08.000+00:00","We were facing some issues with faulty file upload in Chatwoot. How did we take this issue, and how did we finally fix it? Here's a full account!",null,[19],{"id":20,"name":21,"slug":22,"profile_image":23,"cover_image":17,"bio":17,"website":17,"location":17,"facebook":17,"twitter":17,"meta_title":17,"meta_description":17,"url":24},"628363ae4b8f26503f72dedb","Tejaswini Chile","tejaswini","https://www-internal-blog.chatwoot.com/content/images/2022/07/tejaswini.jpeg","https://www-internal-blog.chatwoot.com/author/tejaswini/",[26,30],{"id":27,"name":28,"slug":28,"description":17,"feature_image":17,"visibility":12,"og_image":17,"og_title":17,"og_description":17,"twitter_image":17,"twitter_title":17,"twitter_description":17,"meta_title":17,"meta_description":17,"codeinjection_head":17,"codeinjection_foot":17,"canonical_url":17,"accent_color":17,"url":29},"6118da864b8f26503f72d530","blog","https://www-internal-blog.chatwoot.com/tag/blog/",{"id":31,"name":32,"slug":33,"description":34,"feature_image":17,"visibility":12,"og_image":17,"og_title":17,"og_description":17,"twitter_image":17,"twitter_title":17,"twitter_description":17,"meta_title":35,"meta_description":36,"codeinjection_head":17,"codeinjection_foot":17,"canonical_url":37,"accent_color":17,"url":38},"6308cfbf730b190d1c2d7f42","Engineering","engineering","We're always experimenting with new features and complex solutions, as well as improving our existing offerings. Read about how we do it, directly from Chatwoot engineers.","Engineering Blog | Learnings, tips, stories from Chatwoot Engineers","We're always experimenting with new features, & complex solutions, & improving our existing offerings. Read about how we do it – written by our engineers.","https://www.chatwoot.com/tags/engineering","https://www-internal-blog.chatwoot.com/tag/engineering/",{"id":20,"name":21,"slug":22,"profile_image":23,"cover_image":17,"bio":17,"website":17,"location":17,"facebook":17,"twitter":17,"meta_title":17,"meta_description":17,"url":24},{"id":27,"name":28,"slug":28,"description":17,"feature_image":17,"visibility":12,"og_image":17,"og_title":17,"og_description":17,"twitter_image":17,"twitter_title":17,"twitter_description":17,"meta_title":17,"meta_description":17,"codeinjection_head":17,"codeinjection_foot":17,"canonical_url":17,"accent_color":17,"url":29},"https://www-internal-blog.chatwoot.com/improving-chatwoot-file-upload/",5,true,1775212115374]