Synthesis

Excellent plugin. Have you considered allowing developers to specify whether the stylesheet_link_merged and javascript_include_merged helpers should return merged content in non-production environments? Checking for the presence of a specific environment variable or looking in the development/test/production.rb or environment.rb files would probably suffice.

Another worthwhile enhancement may be to modify the asset helpers to allow for “shorthand” that loads multiple .js/.css files given the name/symbol of their associated parent, merged file as defined in the YAML. An example is shown below.

Given the following snippet from the YAML file:

javascripts: - base: – prototype – effects – controls – dragdrop – application - secondary: – foo – bar

<%= javascript_include_merged :base %> would result in the following markup being generated in non-merging environments:

It would follow that <%= javascript_include_merged :secondary %> would result in the following markup being generated in non-merging environments:

That type of shorthand would be preferable to the current method of specifying the files separately as follows:

<%= javascript_include_merged ‘foo’, ‘bar’ %>

Party foul on me for the earlier formatting snafu. This posting should look better…

Another worthwhile enhancement may be to modify the asset helpers to allow for “shorthand” that loads multiple .js/.css files given the name/symbol of their associated parent, merged file as defined in the YAML. An example is shown below.

Given the following snippet from the YAML file:

javascripts: - base: – prototype – effects – controls – dragdrop – application - secondary: – foo – bar

<%= javascript_include_merged :base %> would result in the following markup being generated in non-merging environments:
<script src='/javascripts/prototype.js' type='text/javascript'></script> <script src='/javascripts/effects.js' type='text/javascript'></script> <script src='/javascripts/controls.js' type='text/javascript'></script> <script src='/javascripts/dragdrop.js' type='text/javascript'></script> <script src='/javascripts/application.js' type='text/javascript'></script>

It would follow that <%= javascript_include_merged :secondary %> would result in the following markup being generated in non-merging environments:
<script src='/javascripts/foo.js' type='text/javascript'></script> <script src='/javascripts/bar.js' type='text/javascript'></script>

That type of shorthand would be preferable to the current method of specifying the files separately as follows:

<%= javascript_include_merged ‘foo’, ‘bar’ %>

Hi Scott,

AssetPackager sounds great! Can’t wait to use this on my own projects.

Thanks for your comment on my site about RoR-Mail. I replied to you over there. I’ll summarize here.

I’ll be putting the RoR-Mail code up in my own svn server, much as you’ve done here for your own projects. My server has plenty of unused bandwidth. And I get to learn more about SVN that way. ;-)

You are welcome to join my project once it’s up. Contact me through the email above.

Thanks for building the plugin—it’s working great. I’m with Rich Wertz though, and think the syntax he proposes makes for an understandable and simple shorthand.

Well Rich, Nick, I agree, and guess what? It’s now in there – include assets by package name. In development, the helper will include all uncompressed asset files. In production, it includes the single compressed asset.

Great work and thanks again for such an important contribution to the Rails community and Web 2.0 in general. For JS-heavy apps, your plugin is a lifesaver.

Prior to giving AssetPackager a go, I was using Dojo’s JS compressor with custom shell scripts/capistrano.

I have 1,600+ lines of custom javascript in my web application (I.e. excluding the RoR base js libs). The integrated JS compressor didn’t work with my code so I made a few modifications to AssetPackager to use Dojo’s JS compressor and everything is working great now.

The change was pretty minimal. I couldn’t find your email or a contact form to send further details, but you have my email with this post now so email me if you’re interested.

Nice work!

For Capistrano you should be able to invoke the other Rake task directly:

task :after_symlink, :roles => [:app] do
  Rake::Task["asset:packager:build_all"].invoke
end

There is some bug with file extension if stylesheet or js named with dot, for ex. admin.base.css

assert_packege_helper.rb line 42

# this code add ext only if no dot there
source << ".#{ext}" unless source.split("/").last.include?(".")

But in assert_packege.rb line 139

# ext added without condition about dot
File.open("#{@asset_path}/#{s}.#{@extension}", "r") { |f| 

simple patch for assert_packege_helper.rb line 42

source << ".#{ext}" unless source.split("/").last =~ /\.(js|css)$/

Using edge rails, I’ve just started getting the following error:

uninitialized constant Synthesis::AssetPackageHelper::AssetPackage

#{RAILS_ROOT}/vendor/rails/activesupport/lib/active_support/dependencies.rb:197:in `load_missing_constant’ #{RAILS_ROOT}/vendor/rails/activesupport/lib/active_support/dependencies.rb:275:in `const_missing’ #{RAILS_ROOT}/vendor/plugins/asset_packager/lib/synthesis/asset_package_helper.rb:18:in `javascript_include_merged’

Looks like it may have something to do with the new dependencies code in edge.

Oops – looks like that messed up formatting somewhwat… sorry!

When I installed, I made a booboo: I installed to assert_package, not asset_package (note the extra R). The build_all Rake task failed saying it couldn’t find jsmin.rb.

The following patch corrects the problem.

Index: lib/synthesis/asset_package.rb
===================================================================
--- lib/synthesis/asset_package.rb      (revision 26)
+++ lib/synthesis/asset_package.rb      (working copy)
@@ -151,7 +151,7 @@
       end

       def compress_js(source)
-        jsmin_path = "#{RAILS_ROOT}/vendor/plugins/asset_packager/lib" 
+        jsmin_path = File.join(__FILE__, '..', 'jsmin.rb')
         tmp_path = "#{RAILS_ROOT}/tmp/#{@target}_#{revision}" 

         # write out to a temp file

Ignore my comment on invoking it directly…

This plugin works fine on my development machine which is running 1.8.2 and the latest frozen rails.

However, on my hosting server, which is running ruby 1.8.4, my site does not work.

Since I have a frozen version of rail under the vendor directory, it does not matter which version rails my hosting server is using, right?

Below is the error message I get.

Thanks

=> Booting WEBrick… ./script/../config/../vendor/rails/activesupport/lib/active_support/dependencies.rb:215:in `load_missing_constant’: uninitialized constant ActionView::Helpers::JavaScriptHelper::PrototypeHelper (NameError) from ./script/../config/../vendor/rails/activesupport/lib/active_support/dependencies.rb:296:in `const_missing’ from ./script/../config/../vendor/rails/actionpack/lib/action_view/helpers/javascript_helper.rb:45 from /usr/lib/site_ruby/1.8/rubygems/custom_require.rb:21:in `require’ from ./script/../config/../vendor/rails/activesupport/lib/active_support/dependencies.rb:331:in `require’ from ./script/../config/../vendor/rails/actionpack/lib/action_view/helpers/prototype_helper.rb:1 from /usr/lib/site_ruby/1.8/rubygems/custom_require.rb:21:in `require’ from ./script/../config/../vendor/rails/activesupport/lib/active_support/dependencies.rb:331:in `require’ from ./script/../config/../vendor/rails/actionpack/lib/action_view/base.rb:205:in `load_helpers’ ... 15 levels… from /usr/lib/site_ruby/1.8/rubygems/custom_require.rb:21:in `require’ from ./script/../config/../vendor/rails/activesupport/lib/active_support/dependencies.rb:331:in `require’ from ./script/../config/../vendor/rails/railties/lib/commands/server.rb:39 from ./script/server:3

Timur – your patch has been accepted, asset_packager now tolerates periods in asset names.

Francois – your patch caused my compressed javascript asset files to be blank. :) But, I did add your Capistrano “after_code_update” task to the docs.

Also, before the compressed assets were getting the subversion revision for the entire repository. Now they get the last modified revision possible for each file. Now it should work as intended – no revision stamp change unless the source files have been updated!

Hi, great plugin!

Do you foresee any future configuration for stylesheets having different media (ie print)?

Hi, great plugin!

Do you foresee any future configuration for stylesheets having different media (ie print)?

Grant, you can already configure different media for compressed stylesheets:

<%= stylesheet_link_merged :base, 'media' => 'print' %>

Is it just me, or does this not work? Nothing fails explicitly, it just doesn’t go into production mode when I switch the environment. I’m running mongrel, but I assume lots of other are too.

Sorry to sound harsh, I greatly appreciate this plugin. Working with a stylesheet per partial makes a lot of sense to me. JS too. It’s too easy to end up with one monolithic js script otherwise, and js seems to be the language future maintainers are the least likely to understand.

Hi, Scott, This plugin looks very helpful for improving rails performance. However, I have some questions, can you help?

1) I can’t get the rake task to work.

rake asset:packager:create_yml
rake aborted!
Don't know how to build task 'asset:packager:create_yml'

Then I tried:

rake -f asset_packager_tasks.rake  asset:packager:create_yml
rake aborted!
Don't know how to build task 'environment'

2) Will this work on tiny_mce? tiny_mce comes with a whole directory with images, style sheets. There is a nice plugin: http://garbageburrito.com/blog/entry/89 However, it seems compressing the files on the fly, which might not be the fast way.

Thanks a lot!

Great plugin. Does it work when you have more than one dot in the filename? For example ‘moo.fx.js’. I can’t get that to work.

If you’re using prototype and javascript crashes after it’s been packaged, edit line 846 of prototype.js:

if (params && /Konqueror|Safari|KHTML/.test(navigator.userAgent)) params += '&_=';

Notice the semicolon at the end: you must add this for prototype to work with asset_packager! At least in my experience.

a small, tiny, minor suggestion… :-)

on line 118 in lib/synthesis/asset_package.rb, direct errors on the svn info call to /dev/null for those of us not using subversion like so:

`svn info #{path} 2>/dev/null`[/Last Changed Rev: (.*?)\n/][/(\d+)/].to_i

That way we don’t see svn griping when running the rake task… Of course my suggestion’s not Windows friendly but could be made so rather easily I think…

Thanks much for a wonderful plugin!

I had a space in my RAILS_ROOT (Windows user), there are a couple of places where that creates problems; I just wrapped the path in quotes.

Also, your CSS compressor doesn’t correctly handle @import statements; they need to be moved to the top.

Finally, I didn’t like that the app crashed in production if the assets weren’t built; I modified AssetPackager#current_file to return nil if the requested package wasn’t built.

diff -ur asset_packager/lib/synthesis/asset_package.rb asset_packager2/lib/synthesis/asset_package.rb
--- asset_packager/lib/synthesis/asset_package.rb       2007-05-15 19:26:47.925000000 -0500
+++ asset_packager2/lib/synthesis/asset_package.rb      2007-05-15 19:12:52.165000000 -0500
@@ -23,7 +23,7 @@
       package_names = Array.new
       sources.each do |source|
         package = find_by_target(asset_type, source) || find_by_source(asset_type, source)
-        package_names << (package ? package.current_file : source)
+        package_names << (package && package.current_file || source)
       end
       package_names.uniq
     end
@@ -80,7 +80,9 @@
     end

     def current_file
-      Dir.new(@asset_path).entries.delete_if { |x| ! (x =~ @match_regex) }.sort.reverse[0].chomp(".#{@extension}")
+      returning Dir.new(@asset_path).entries.delete_if { |x| ! (x =~ @match_regex) }.sort.reverse[0] do |s|
+        s.chomp!(".#{@extension}") if s
+      end
     end

     def build
@@ -115,7 +117,7 @@
       def get_file_revision(path)
         if File.exists?(path)
           begin
-            `svn info #{path}`[/Last Changed Rev: (.*?)\n/][/(\d+)/].to_i
+            `svn info "#{path}"`[/Last Changed Rev: (.*?)\n/][/(\d+)/].to_i
           rescue # use filename timestamp if not in subversion
             File.mtime(path).to_i
           end
@@ -158,7 +160,7 @@
         File.open("#{tmp_path}_uncompressed.js", "w") {|f| f.write(source) }

         # compress file with JSMin library
-        `ruby #{jsmin_path}/jsmin.rb <#{tmp_path}_uncompressed.js >#{tmp_path}_compressed.js \n`
+        `ruby "#{jsmin_path}/jsmin.rb" <"#{tmp_path}_uncompressed.js" >"#{tmp_path}_compressed.js" \n`

         # read it back in and trim it
         result = "" 
@@ -178,7 +180,9 @@
         source.gsub!(/\n$/, "")            # remove last break
         source.gsub!(/ \{ /, " {")         # trim inside brackets
         source.gsub!(/; \}/, "}")          # trim inside brackets
-        source
+       imports = []
+       source.gsub!(/(@import.*;)/) { |m| imports.push(m); "" }
+       imports.join(" ") + source
       end

       def get_extension
@@ -197,4 +201,4 @@
       end

   end
-end
\ No newline at end of file
+end

What do you think?

Nick

I’ve been using Asset Packager for two weeks now, and it’s working great. For my use, there’s only one issue: no svn integration.

‘rake asset:packager:build_all’ successfully adds new files locally and to svn, but only locally deletes from svn. As a result, I have to manually remove the old files from svn.

I had a brief look at the code but couldn’t figure out how to fix this issue. I’ll give it another shot later on.

@Xin – You shouldn’t be committing the packages to svn. They should be thought of as temporary files, built from your original javascript and css files. No need to commit them, as they can always be rebuilt.

Fantastic plugin, Scott, really helpful. My apologies in advance for this long post…

I find it helpful in my current project to have several subdirectories under stylesheets, to hold different modules and parts of the site. I like to break things up so I don’t need to wade through 100s of lines of CSS to find what I want. So, I had asset_packages.yml looking like…

stylesheets: 
- shared/package:
  - shared/tables
  - shared/forms
  - shared/datepicker
  - shared/pagination
- admin/package:
  - admin/style
  - admin/navigation

The packages are called shared/package rather than just shared because the packages must be built in the same directory as the source, otherwise image references break. So I ran rake asset:packager:build_all and all looked well, everything in its right place.

Problem is, the packages are not loaded properly in production mode because of the subdirectories being ignored. So I made a patch that lets you write like this:

stylesheets: 
- shared/package:
  - tables
  - forms
  - datepicker
  - pagination
- admin/package:
  - style
  - navigation

and asset_packager will interpret the source paths relative to the target directory for each package – like I said, you should generate the package in the same dir as all its source. This patch then fixes everything up in development and production. I’ll just paste the rewritten methods rather than diff file…

def self.sources_from_targets(asset_type, targets)
  source_names = Array.new
  targets.each do |target|
    package = find_by_target(asset_type, target)
    source_names += (package ? package.sources.collect do |src|
      package.target_dir.gsub(/$/, '\1/') + src
    end : target.to_a)
  end
  source_names.uniq
end

...

attr_accessor :asset_type, :target, :target_dir, :sources

def initialize(asset_type, package_hash)
  target_parts = /((.*)\/)?([\/]+)$/.match(package_hash.keys.first).to_a
  @target_dir = target_parts[2].to_s
  @target = target_parts[3].to_s
  @sources = package_hash[package_hash.keys.first]
  @asset_type = asset_type
  @asset_path = ($asset_base_path ? "#{$asset_base_path}/" : "#{RAILS_ROOT}/public/") +
      "#{@asset_type}#{@target_dir.gsub(/(.+)$/, '/\1')}" 
  @extension = get_extension
  @match_regex = Regexp.new("\\A#{@target}_\\d+.#{@extension}\\z")
end

def current_file
  @target_dir.gsub(/^(.+)$/, '\1/') +
      Dir.new(@asset_path).entries.delete_if { |x| ! (x =~ @match_regex) }.sort.reverse[0].chomp(".#{@extension}")
end

You can still write the yml file for single directories just as before. Any chance you could include this?

Just realised Textile garbled some of my regexps. Email me and I’ll send you the patch if you like.

hello, i’m quite new to Ruby on Rails, so I’m sorry if this is a newbie question.

I have rails apps currently running in production, but sometimes I still need to make small changes to my css or javascript. From the explanation i read, it’s said that the rake commands build_all need to be executed only once, during development. But it looks like the changes I made didn’t work, even though i’ve restarted my server, unless i re-execute the rake command again. Is that what it’s supposed to be, or am I made a mistake here?

Thanx for any advice you can give me. :)

The rake command build_all needs to be run only once when deploying to your production server. Not in development.

If you make changes to your originals, you’ll need to re-execute the rake task again.

Scott: OMFG I feel so stupid right now – forget about my mail :)

Anyways, for capistrano 2 the task should be changed to:

namespace :deploy do
 desc "Create asset packages for production" 
 task :after_update_code, :roles => [:web] do
   run <<-EOF
     cd #{release_path} && rake RAILS_ENV=production asset:packager:build_all
   EOF
 end
end

Seems to work well that way – at least for me :).

i’ve added an @@options variable to AssetPackage class to control if merging should be done or not. it defaults to RAILS_ENV == 'production'. like this i can force asset_packager to serve the merged version or not. i needed this to test everything in development. also, after i tried it first time in production, the css didnt work properly, but i wanted to keep the <%= stylsheet_link_merged :base %>, and just force it to link to the separated files, until i fix the problem. here’s the diff.

thanks :) for asset_package.rb:

--- asset_package.rb
+++ (clipboard)
@@ -1,6 +1,11 @@
 module Synthesis
   class AssetPackage
     # class variables
+    @@options = {
+      :merge => (RAILS_ENV == "production")
+    }
+    cattr_accessor :merge
+
     @@asset_packages_yml = $asset_packages_yml || 
       (File.exists?("#{RAILS_ROOT}/config/asset_packages.yml") ? YAML.load_file("#{RAILS_ROOT}/config/asset_packages.yml") : nil)

for asset_packager_helper.rb

--- asset_package_helper.rb
+++ (clipboard)
@@ -13,7 +13,7 @@
       end

       sources.collect!{|s| s.to_s}
-      sources = (RAILS_ENV == "production" ? 
+      sources = (AssetPackage.options[:merge] ? 
         AssetPackage.targets_from_sources("javascripts", sources) : 
         AssetPackage.sources_from_targets("javascripts", sources))

@@ -24,7 +24,7 @@
       options = sources.last.is_a?(Hash) ? sources.pop.stringify_keys : { }

       sources.collect!{|s| s.to_s}
-      sources = (RAILS_ENV == "production" ? 
+      sources = (AssetPackage.options[:merge] ? 
         AssetPackage.targets_from_sources("stylesheets", sources) : 
         AssetPackage.sources_from_targets("stylesheets", sources))

For some reason the post James did with subdirectories did not work for me. Instead of debugging I wrote something that worked for my situation. I only needed to support using a subdirectory to one level. This change only affects one function and lets the yaml file stay the same (with subdirectory in the package name and file names). When referencing the file from the merged function just use ‘subdirectory/filename’. Someone could definitely rewrite this to support n-levels of nesting if they find it necessary.

    -- asset_package.rb

    def current_file
      subdirectory = @target.split(/\//)
      if subdirectory.length == 2 then
        new_match_regex = Regexp.new(@match_regex.source.gsub("#{subdirectory[0]}\/", ""))
        file = Dir.new("#{@asset_path}/#{subdirectory[0]}").entries.delete_if { |x| ! (x =~ new_match_regex) }.sort.reverse[0].chomp(".#{@extension}")
        "#{subdirectory[0]}/#{file}" 
      else
        Dir.new(@asset_path).entries.delete_if { |x| ! (x =~ @match_regex) }.sort.reverse[0].chomp(".#{@extension}")
      end
    end

Love this plugin!

I often run into problems with jsmin when minifying certain libraries (if I recall correctly some components of YUI didn’t minify properly). Also, there is this bit in the code that messes up most of YUI’s CSS files (since it uses CSS hacks):

source.gsub!(/\/\\\/ /, "") if @@remove_css_comments # remove comments - caution, might want to remove this if using css hacks

Usually I just comment out the stuff I don’t need locally (I use Piston to manage my plugins). But it got kinda troublesome especially when I forget to do so on new Rails apps.

I’ve added a patch: http://pastie.caboo.se/75340 that would allow you to do things like this in your environment.rb (or config/initializers for Edge Rails users):

Synthesis::AssetPackage.compress_js = false
Synthesis::AssetPackage.compress_css = true
Synthesis::AssetPackage.remove_css_comments = false

I hope you’d take a look and commit it (or even better, do something better and then commit your improvements!)

Thanks again for a wonderful plugin, Scott!

Hey Scott,

Somehow the patch in the previous Pastie didn’t work as advertised (I swear it was working once for me). It’s to do with when the class attributes are loaded (basically the config options in environment.rb didn’t get recognized because of the order the variables are declared or some reason). It’s boggling my brain at the moment why the class attributes get set as they do (meaning the @@compress_js = true in the AssetPackage class itself overrides any Synthesis::AssetPackage.compress_js = false setting in environment.rb).

So I reverted back to my uglier patch: http://pastie.caboo.se/75578 that I’m quite sure would work.

The build_all doesn’t appear to handle DOS carriage returns very well. I kept getting the exception: “Unterminated string literal”... couldn’t figure it out until I guessed that there might be a file with DOS-based line endings.

Ran dos2unix on all the JS and everything was happy.

Hey this plugin is cool!

I really like it in theory, however in practicality, the regular expressions don’t quite cut the mustard for me (i must be missing some semi-colons somewhere)

This would be way, waaaaaaaay cooler (and I would definitely use it) if you had support for an engine based javascript optimizer, ie. shrinksafe: http://alex.dojotoolkit.org/shrinksafe/

Even if that introduced a Java Runtime dependency, it’s still definitely worth it to get at the reliability of the Rhino interpreter which will appropriately manage missed semi-colons etc, more thouroughly obfuscate logic with variable renaming & deliver better performing file size reductions.

Some Stats on my applications javascript reduction: 161kb compiled – Ruby JavaScript Minifier 146kb compiled – Shrinksafe

You could maybe even avoid the Java dependency if you hosted the logic as a web service that was posted to during the capistrano deployment.

Also have you seen: http://blog.jcoglan.com/2007/05/26/make-capistrano-compress-your-javascript-and-css-automatically/

Gzipping your output for supported browsers would be another really nice extension to your plugin =]

Hi Scott,

Great plugin! Thank you! I was wondering if support for stylesheets & javascript in subdirectories will be added?

hey, we used this, it’s great. But we found that it will remove ”;” before “}” and it will make css file broken on Safari2. after remove code : source.gsub!(/; \}/, “}”) # trim inside brackets everything works well.

The comment by ‘yoav’ was getting right at what I needed. Its really important to be able to test how this loads your JS and CSS in dev, (staging, if you have one), and production environments. My first quick reaction was to just change the code in the plugin to hardcode staging and production for my use… But that wouldn’t be nice…

Also, the patch provided in the comments didn’t actually work for me, and I had to scratch my head a few times to figure out cattr_accessor which I had not used before.

So here is a patch which works for me (just slight modification from above), and you can see how exactly you can use it in the comments in the patch which can be found here:

http://pastie.caboo.se/81657

To apply the patch, copy this file into your asset_packager dir in a file called config.patch and run:

patch -p0 config.patch

To use it in your environment.rb file add something like:

Synthesis::AssetPackage.merge = ( RAILS_ENV "development" || RAILS_ENV “staging” || RAILS_ENV == “production”)

Thank you swing1979@gmail.com! That comment helped me out a lot. Indeed, the CSS compressor is hosing CSS files for Safari2.

I encountered two major issues using this:

1. Spaces in RAILS_ROOT breaks it (as stated by Nick above)

2. The parsing of the revision number doesn’t work in non-English svn configurations. Using the—xml parameter to “svn info” would make this way more robust, as the xml format is always in English (and it would make it parsable by REXML as well).

Example:

xml = REXML::Document.new `svn info --xml "#{path}"`
xml.elements["info/entry/commit"].attributes["revision"].to_i

(Also note the ”” surrounding the path variable. ;)

A minor concern is regarding the limited possibilities of configuring when the merged files should be used and not. Various solutions are presented above.

Do you have any plans on releasing a new version with these suggestions/fixes in place? ;-)

Ran into some JavaScript errors when I deployed with the packaged jsmin.rb. I downloaded the updated version from http://www.crockford.com/javascript/jsmin.html and all works well.

I’m running into a problem during initial setup:

./script/plugin install http://sbecker.net/shared/plugins/asset_packager
rake asset:packager:create_yml

leads to:

rake aborted!
no such file to load -- synthesis/asset_package

Well, my vendor/plugins/asset_packager/lib does contain a subfolder named synthesis with asset_package.rb in there, so I have no idea why it’s complaining. Any ideas?

okay, found out what the problem mentioned above was: I didn’t include the plugin in my environment.rb…

I noticed that if you have files called base_999.css and base_1001.css, the plugin will serve base_999

here is the fix:

    def current_file
      Dir.new(@asset_path).entries.delete_if { |x| ! (x =~ @match_regex) }.sort_by{|a|a.split('_').last.to_i}.reverse[0].chomp(".#{@extension}")
    end

i am facing a problem in merging as i have js files in various directories and subdirectories can u please guide me on this how we can create the tree structure fro files in sub directories . like in javascripts / yui / yahoo i have yahoo.js what will be the structure fro these files and are all files called secondary .

my email :- raghubetter@gmail.com

Hey finally i soughted out using YUI compressor ! works fine Thanks

I love this plugin, great work!

But I have been having a problem. Sometimes when I package all my Javascript together and deploy some components on my site no longer work. I will get the error message “Ajax is undefined”. Sometimes this can be fixed by moving Prototype.js outside of the asset packager, but sometimes it doesn’t fix it. Any ideas?

What a great plugin, might I ask for a support of including directories options (to merge all files within a directory)

Thanks again

I’m surprised no one else has mentioned this but Timur Vafin’s patch from August 4, 2006 breaks image_tag because it only looks for .js or .css at the end of an asset before deciding whether to append the default ext or not. In the case of an image that line will cause .png to be appended to the image path even if .gif is already there. A quick fix is to patch asset_package_helper.rb:42 again like so:

@source << ".#{ext}" unless source.split("/").last =~ /\.(js|css|gif|jpg|png)$/@

However that still seems brittle and perhaps there is a better solution.

I’ve modified a little asset_packager’s helpers so they won’t create include tags for files that don’t exist, you can read about my way of using asset_packager here

oh and Marshall: Check if Firebug shows any js-related errors, in prototype 1.5. there are some problems with lines having no semicolon at the end, which breaks compressed js code.

using capistrano 2.0 hooks seem better

desc "build the minified javascript and css files using asset packager" 
after "deploy:update_code" do
  run "cd #{release_path} && rake RAILS_ENV=production asset:packager:build_all" 
end

If you are defining :rake in your deploy.rb, then you may need to modify the :after_update_code task to something like this (notice the #{rake}):

task :after_update_code, :roles => [:app] do
  run "cd #{release_path} && #{rake} asset:packager:build_all RAILS_ENV=production" 
end

Before I did that, I was picking up a “bad” version of rake and getting errors saying “Cannot find gem for Rails ~>1.2.3.0”.

While trying out edge rails on a new project I found that using image_tag was failing, claiming that there weren’t enough variables (2 for 3).

After taking a poke about it turns out that image_tag was using the compute_public_path method from asset_packager instead of the rails one.

I’ve put the diff up on pastie

when using shrinksafe i get a better compression, and jsmin on top of it to compress new lines… shrinksafe also verifies the compression instead of doing a blind one, and will also compress local variable names…

open asset_package.rb, comment line 161 and add this line under it (assuming java is in /usr/bin/java):

`/usr/bin/java -jar #{jsmin_path}/custom_rhino.jar -c #{tmp_path}_uncompressed.js | ruby #{jsmin_path}/jsmin.rb > #{tmp_path}_compressed.js \n`

make sure to download custom_rhino.jar from dojo (http://dojotoolkit.org/docs/shrinksafe) and put it inside RAILS_ROOT/vendor/plugins/asset_packager/lib

Colin: Had the same problem, thanks for the patch!

Note there seems to be a missing semicolon in the Rails 1.2.4 cut of prototype.js—making the squished .js set browser-unhappy. This was my fix.

--- prototype.js~       2007-10-06 21:50:05.000000000 -0700
+++ prototype.js        2007-10-07 16:58:28.000000000 -0700
@@ -843,7 +843,7 @@
     }

     params = Hash.toQueryString(params);
-    if (params && /Konqueror|Safari|KHTML/.test(navigator.userAgent)) params += '&_='
+    if (params && /Konqueror|Safari|KHTML/.test(navigator.userAgent)) params += '&_=';

     // when GET, append parameters to URL
     if (this.method == 'get' && params)

Hey Colin, thanks a lot for that quick fix. It would be great if it was integrated into asset_packager some time soon

Can anyone confirm whether or not this plugin currently supports assets under subdirectories (e.g. public/javascripts/calendar_date_select/file.js)?

Thanks :)

@chris: I actually sent Scott a patch for doing just that a while back but I don’t know what happened to it. He was on the fence about including it and it’s not gone into the trunk. Drop me a line at my site and I can send you my version of the plugin.

Chris & James – I finally got around to applying that patch. Subdirectories a-plenty! Sorry it took so long.

Holy crap, theres a lot of activity in the comments here. Hadn’t checked this in a while. I need to create a forum for AssetPackager.. and a core team, and an irc channel, and and.. :)

Setting up with .js & .css files in more subdirectories. I have installed CalendarDateSelect and it creates its own subdirectories for usage. How to we specify these files. I knoticed when I ran the first rake asset:packager:create_yml

It does not seem to find these files under the subdirectory. How do I tell it to find these?

Setting up with .js & .css files in more subdirectories. I have installed CalendarDateSelect and it creates its own subdirectories for usage. How to we specify these files. I knoticed when I ran the first rake asset:packager:create_yml

It does not seem to find these files under the subdirectory. How do I tell it to find these?

So I set up my yml file like this: @- javascripts: - base: – jquery – corner/jquery.corner – chili/jquery.chili – jquery.form – jquery.dimensions – jquery.compatibility - edit_form: – ui/ui.calendar – jquery.autocomplete – wymeditor/jquery.wymeditor - app: – application stylesheets: - base: – style - edit_form: – ui.calendar – wymeditor/screen@

It works fine but when run the rake task and try and use it in the production environment I have errors. The first is in the edit_form.js file the packed version of wymeditor. The second and more serious issue is that the edit_form.css file only has two classes and didn’t include any of the rest. It just didn’t pack nearly all the styles. Any ideas?

Scott, FYI, asset_packager worked fine for me in Rails 1.2.3, but in Rails 1.2.5 I get this error on every image if this plugin is installed:

/Library/Ruby/Gems/gems/actionpack-1.13.5.7919/lib/action_view/helpers/asset_tag_helper.rb:327:in `compute_public_path’ /Library/Ruby/Gems/gems/actionpack-1.13.5.7919/lib/action_view/helpers/asset_tag_helper.rb:327:in `image_path’ /Library/Ruby/Gems/gems/actionpack-1.13.5.7919/lib/action_view/helpers/asset_tag_helper.rb:359:in `image_tag’

It’s very odd. If I remove the plugin, the issue goes away. Not sure if the bug is with you, me, or Rails. Anyway, thought you might be interested.

Scott, FYI, asset_packager worked fine for me in Rails 1.2.3, but in Rails 1.2.5 I get this error on every image if this plugin is installed:

/Library/Ruby/Gems/gems/actionpack-1.13.5.7919/lib/action_view/helpers/asset_tag_helper.rb:327:in `compute_public_path’ /Library/Ruby/Gems/gems/actionpack-1.13.5.7919/lib/action_view/helpers/asset_tag_helper.rb:327:in `image_path’ /Library/Ruby/Gems/gems/actionpack-1.13.5.7919/lib/action_view/helpers/asset_tag_helper.rb:359:in `image_tag’

It’s very odd. If I remove the plugin, the issue goes away. Not sure if the bug is with you, me, or Rails. Anyway, thought you might be interested.

Hey is there a solution to get this working with the multiple_asset_hosts plugin? http://spattendesign.com/2007/10/24/setting-up-multiple-asset-hosts-in-rails

The multiple_assets code is now apart of rails 2.0, it was extracted into that plugin for use in rails

The compute_public_path in asset_package_helper doesn’t play with the new compute_public_path in asset_tag_helper for asset hosts. Anyone have a fix for this?

Hey is there a solution to get this working with the multiple_asset_hosts plugin? http://spattendesign.com/2007/10/24/setting-up-multiple-asset-hosts-in-rails

The multiple_assets code is now apart of rails 2.0, it was extracted into that plugin for use in rails

The compute_public_path in asset_package_helper doesn’t play with the new compute_public_path in asset_tag_helper for asset hosts. Anyone have a fix for this?

(Sorry if my comment posted twice, this comment form needs some tweaking!)

Hello. I’m getting an error using asset packager on a blank rails project (and my own rails project. Here are some details:

OS X 10.4 Ruby 1.8.6 Rake 0.7.3 Rails 1.2.4 and 1.2.5

Create a blank rails project, install this plugin, run asset:packager:create_yml and I get the following. Any ideas?

me@computer:~/eng/foo:1>rake asset:packager:create_yml—trace (in /Users/jjb/eng/foo)

• Invoke asset:packager:create_yml (first_time)
• Execute asset:packager:create_yml rake aborted! undefined method `log’ for Synthesis::AssetPackage:Class /Users/jjb/eng/foo/config/../vendor/plugins/asset_packager/tasks/../lib/synthesis/asset_package.rb:87:in `create_yml’ /Users/jjb/eng/foo/config/../vendor/plugins/asset_packager/tasks/asset_packager_tasks.rake:18 /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/rake.rb:392:in `call’ /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/rake.rb:392:in `execute’ /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/rake.rb:392:in `each’ /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/rake.rb:392:in `execute’ /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/rake.rb:362:in `invoke’ /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/rake.rb:355:in `synchronize’ /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/rake.rb:355:in `invoke’ /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/rake.rb:1739:in `top_level’ /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/rake.rb:1739:in `each’ /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/rake.rb:1739:in `top_level’ /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/rake.rb:1761:in `standard_exception_handling’ /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/rake.rb:1733:in `top_level’ /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/rake.rb:1711:in `run’ /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/rake.rb:1761:in `standard_exception_handling’ /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/rake.rb:1708:in `run’ /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/bin/rake:7 /usr/local/bin/rake:16:in `load’ /usr/local/bin/rake:16

Can the compression be optional, maybe add a config?

Scott,

I just discovered and fixed a file descriptor leak in asset_package.rb. We’ve been using this plugin happily in production for a long time. Thank you so much for making it available. Recently, as traffic on our site has been increasing, we started getting occasional process failures with the following message from Mongrel:

“Mongrel timed out this thread: too many open files”

I tracked it down to the code in asset_package.rb that does this:

Dir.new(“xyz”).entries …

Each time this code is run it leaks a file descriptor.

The problem goes away if you change that code to:

Dir.entries(“xyz”) ...

Thanks again for your wonderful plugin!

Tyler

If anyone is seeing an error like the following:

$ rake RAILS_ENV=production asset:packager:build_all --trace
(in /export/sites/beta.femmetastics.com/beta/releases/20071109172808)
rake aborted!
uninitialized constant Synthesis::AssetPackage::YAML

Just add a quick

require 'yaml'

to the top of lib/syntheis/asset_package.rb, and that should do the trick.

The issue with the undefined method is caused by the plugin expecting the log method to exist (which exists in edge). My work-around was just to create a blank log method at the top of asset_package.rb

def log (msg)

end

You will not get logs but the program will work. You can add something like puts msg as the body of the log method, how you want to handle that is up to you.

May I suggest a :license option?

Motivation:

Licenses of prototype.js, calendar.js etc should be included to respect the original author’s conditions for distribution.

Usage:

Suppose foo.js and bar.js have licensing conditions attached.

Copy the foo and bar licenses to the foobar_license.txt file, with additional pointers/explanation as appropriate.

In the view, add another parameter,

<%= javascript_include_merged :foo, :bar, :license => ‘foobar_license.txt’ %>

Intended Result

The license-file is prepended to the compressed file, as a javascript comment.

Thanks a lot for the plugin. I’m planning to use it with YahooUI’s compressor, and code from http://www.kangarooit.com/developer_blog section “Asset Packager with Custom Rhino for super optimized JavaScript”

Stephan

Anyone know why I might be getting a ” private method `chomp’ called for nil:NilClass” error where I use the stylesheet_link_merged helper ?

I figured it out, the asset packages weren’t getting built by my deploy script. doh!

Is asset_packager integrated into Rails 2? If so, how do I use it? Simply deleting from the plug_in directory didn’t help a lot.

Thanks, Amir

Thanks, Dan! Frustrating.

Anyone who’s getting this error…

rake asset:packager:create_yml
rake aborted!
Don't know how to build task 'asset:packager:create_yml'

might find they need to

rake asset:packager:build_all

first.

Anyone getting this error…

Execute asset:packager:create_yml rake aborted! undefined method `log’ for Synthesis::AssetPackage:Class

should follow Dan’s helpful instructions above and add the following empty method to vendor/plugins/asset_packager/lib/synthesis/asset_package.rb

def log (msg)

end

hi, i upgraded to rails 2.0 and i wanted to use multiple asset hosts like this: config.action_controller.asset_host=”http://assets%d.example.com”

to use it with assetpackager you have to update asset_packager/lib/synthesis/asset_package_helper.rb with the new version from rails sources from http://dev.rubyonrails.org/browser/trunk/actionpack/lib/action_view/helpers/asset_tag_helper.rb?rev=6164

and just replaced compute_public_path method and added rewrite_asset_path!(source) and compute_asset_host methods from the rails sources.

Thats all, now it runs on 4 assets hosts, great and even faster then before …

Asset packager is a fantastic plugin. Per a comment above (about Dir.entries instead of Dir.new(...).entries, couldn’t the following code be further simplified from this:

    def current_file                                                                                                                                                                             
      @target_dir.gsub(/^(.+)$/, '\1/') +                                                                                                                                                        
        Dir.new(@asset_path).entries.delete_if { |x| ! (x =~ @match_regex) }.sort.reverse[0].chomp(".#{@extension}")                                                                                 
    end

To this (grab the last item instead of reversing the list to grab the [0] element…also do x !~ regex rather than ! ( x =~ regex)):

    def current_file                                                                                                                                                                             
      @target_dir.gsub(/^(.+)$/, '\1/') +                                                                                                                                                        
        Dir.new(@asset_path).entries.delete_if { |x| x !~ @match_regex}.sort.last.chomp(".#{@extension}")                                                                                 
    end

Also, if I am reading the code correctly, with every page serve, the stylesheets or javascripts directory are being read for the most recent version of the file listed in the yaml file. Wouldn’t it be better, once the filename/path has been determined, to cache it in a class variable (since files are not being dynamically added to the assets directories?) Maybe like this:

    @@cached_asset_file_paths = {:css => {}, :js => {}}

    def current_file
      unless @@cached_asset_file_paths[@extension.to_sym].has_key?([@target.to_sym])
          @@cached_asset_file_paths[@extension.to_sym][@target.to_sym] = @target_dir.gsub(/^(.+)$/, '\1/') +                                                                                                                                                        
            Dir.new(@asset_path).entries.delete_if { |x| x !~ @match_regex}.sort.last.chomp(".#{@extension}")
      end

      @@cached_asset_file_paths[@extension.to_sym][@target.to_sym]                                                                                 
    end

Thanks again for a great plugin. Definitely makes my site feel snappier.

Fantastic Plugin!

Package Version Numbers!

We ran into an issue when building the asset packages where we modified the asset_packages.yml and hadn’t modified any of the js files. This caused an issue where the version of the packaged files didn’t change thus any browser that had cached the previous version didn’t get the changes.

We therefore have included in the asset_package.rb on revision method:

      def revision
        unless @revision
          revisions = [1]
          revisions << get_file_revision("#{@asset_path}/../../config/asset_packages.yml")
          @sources.each do |source|
            revisions << get_file_revision("#{@asset_path}/#{source}.#{@extension}")
          end
          @revision = revisions.max
        end
        @revision
      end

Notice that we now include the asset_packages.yml version in the max lookup.

I ran into a problem with adding a single javascript file to the asset_packages.yml file. Since the svn version number on the new file was older than the previous version, asset_packager didn’t build a new file, and likewise, browsers wouldn’t know that the file had changed because the version number didn’t change. I think since any change to asset_packages.yml could affect any of the packages, the version number of the asset_packages.yml needs to be included in the calculation. Here’s my patch:

http://pastie.caboo.se/151920

Hello! Thanks for great plugin =) I ran into a problem with javascript_include_merged, which, for some odd reason adds the querystring timestamp to src. Is there any solution, because my JS is over 100 kB and is hard for bandwitch because it’s transferred every time I load the page.

Thanks in advance, Jyrki J.

This seems to fix the problem:

--- a/vendor/plugins/asset_packager/lib/synthesis/asset_package_helper.rb
+++ b/vendor/plugins/asset_packager/lib/synthesis/asset_package_helper.rb
@@ -21,7 +21,10 @@ module Synthesis
         AssetPackage.targets_from_sources("javascripts", sources) : 
         AssetPackage.sources_from_targets("javascripts", sources))

-      sources.collect {|source| javascript_include_tag(source, options) }.join("\n")
+      sources.collect { |source|
+        source = javascript_path(source)
+        content_tag("script", nil, { "type" => "text/javascript", "src" => source }.merge(options))
+      }.join("\n")
     end

     def stylesheet_link_merged(*sources)

Thanks for the great plugin!

One issue that I ran into was the svn command dying when running under mongrel versions > 1.0.2 that are launched as root but drop their privileges. You get this error message in the mongrel log:

svn: Can't check path '/root/.subversion': Permission denied

$HOME is still /root, so svn tries to look for /root/.subversion, which throws an exception (due to permissions), which eventually returns a default revision of 1. In my case I worked around it by adding “—config-dir /does-not-exist” to the svn command.

Thanks again,
Dave

I see that Synthesis::AssetPackage.build_all has a dependency on the “get_file_revision” method which depends on SVN.

Has an update been released to make this Git compatible?

Seems like the compute_public_path has some trouble with file names containing periods (common in jquery libs among others).

The .js either does not get appended (in development) or if u explicit set it in the asset_packages.yml it gets appended even if it should not.

Hackfix:

  line 41: source << ".#{ext}" if File.extname(source).blank?
  === changed to:
  source << ".#{ext}" if File.extname(source).blank? || (!ext.nil? && File.extname(source) != ext)

Seems to fix it.

It works fine but when run the rake task and try and use it in the production environment I have errors. The first is in the edit_form.js file the packed version of wymeditor. The second and more serious issue is that the edit_form.css file only has two classes and didn’t include any of the rest. It just didn’t pack nearly all the styles. Any ideas?

I am getting a “Service Temporarily Unavailable” error for the tracker, thus I’m posting it here:

I recently ran into a subtle bug with compressed style sheets. If you have a comment in a property or value without a following white space like this:

property: /**/value;

the parser will not see the end of this comment and instead strip away everything up until it encounters the next closing comment.

@Joel that worked better for me (based on r57):

source << ".#{ext}" if File.extname(source).blank? && ext || ext =~ /css|js/ && File.extname(source).delete('.') != ext

Is this plugin useful in rails 2? (see http://maintainable.com/articles/rails_asset_cache)? Thanks!

I hacked the merged_file method to support files with with general filename

With this hack, I can put in the full filename in the yml file and the builder doesn’t complain about the file not exists

def merged_file
  merged_file = "" 
  @sources.each {|s| 
    # hack by alex
    fn = "#{@asset_path}/#{s}" 
    fn << ".#{@extension}" unless fn.downcase.split( '.' ).last == @extension.downcase
    File.open( fn, "r") { |f| 
      merged_file += f.read + "\n" 
    }
  }
  merged_file
end

Not sure if asset_packager does this already, but it would be nice to be able to specify a cache subdirectory (a la rails2) so that it is easier to clean up the directory and you lessen the change of accidentally deleting the source file.

Is this compatible with rails 1.1.6? Please do let me know.

When Installed it doesn’t boot my application and i don’t see any error in the log file.

Please do let me know.

Thank you! Paddy

Is this compatible with rails 1.1.6? Please do let me know.

When Installed it doesn’t boot my application and i don’t see any error in the log file.

Please do let me know.

Thank you! Paddy

Does this work in other environments besides “production”? I’d like to enable this in another env, is there an option to do so?

roobnoob, are you using this with a staging environment? And is anyone using Git instead of SVN for the versions the rake task creates?

roobnoob, are you using this with a staging environment? And is anyone using Git instead of SVN for the versions the rake task creates?

I had a little problem with the fact that the rewrite of the compute_public_path function does not take into account the “has_request = @controller.respond_to?(:request)” part from the original function. This made the rails image_path function fail in an email template with inline attachments.

A simple rename of this rewritten compute_public_path function did solve this:

diff --git a/vendor/plugins/asset_packager/lib/synthesis/asset_package_helper.rb b/vendor/plugins/asset_packager/lib/synthesis/asset_package_helper.rb
index 31d0958..df2d3b7 100644
--- a/vendor/plugins/asset_packager/lib/synthesis/asset_package_helper.rb
+++ b/vendor/plugins/asset_packager/lib/synthesis/asset_package_helper.rb
@@ -41,7 +41,7 @@ module Synthesis
     private
       # rewrite compute_public_path to allow us to not include the query string timestamp
       # used by ActionView::Helpers::AssetTagHelper
-      def compute_public_path(source, dir, ext=nil, add_asset_id=true)
+      def asset_packager_compute_public_path(source, dir, ext=nil, add_asset_id=true)
         source = source.dup
         source << ".#{ext}" if File.extname(source).blank? && ext
         unless source =~ %r{^[-a-z]+://}
@@ -55,13 +55,13 @@ module Synthesis

       # rewrite javascript path function to not include query string timestamp
       def javascript_path(source)
-        compute_public_path(source, 'javascripts', 'js', false)       
+        asset_packager_compute_public_path(source, 'javascripts', 'js', false) 
       end

       # rewrite stylesheet path function to not include query string timestamp
       def stylesheet_path(source)
-        compute_public_path(source, 'stylesheets', 'css', false)
+        asset_packager_compute_public_path(source, 'stylesheets', 'css', false)
       end

   end
-end
\ No newline at end of file
+end

Hi Scott. Looks like asset_packager is busted under Rails 2.2:

undefined method `rails_asset_id’ for #

In Rails 2.2, it looks like rails_asset_id became a private method.

The definition of the javascript_include_merged uses a hard coded array as the list of default javascripts. This ignores any new defaults added by the application or plugins via ActionView::Helpers::AssetTagHelper#register_javascript_include_defaults@.I changed the hardcoded array to

ActionView::Helpers::AssetTagHelper.send(:class_variable_get,'@@javascript_expansions'.to_sym)[:defaults] 

to get this working. Sorry no time for a git patch right now.

In the end this didn’t really matter though, as I quit using :defaults at all since using that rails shortcut means that the defaults will not be part of the asset package. If I understand it correctly, if you really want only one javascript file, you need to hand copy the javascripts :defaults into the asset_packages.yml anyhow so it will be up to you to resolve any non-standard defaults there.

Also ran into the 2.2 problem with rails_asset_id, there’s also another problem with relative_url_root

Hi, is it possible to get this to work in development in would save me some serious time?

I had to add require 'synthesis/asset_package' to asset_package_helper.rb as I run without automatic dependency loading.

Also jsmin.rb breaks on javascript like return /regexp/.test(string);, which I hit while processing jquery.tablesorter.js. It seems not to recognise the first / as the start of a regexp.

Check out minify_cache to get a nice monkey patch to rails javascript_include_tag….:cache =>.. for automatically minifying the output.

So, I converted all of the regexp literals in jquery.tablesorter.js from “return /foo/.test(s)” to “return RegExp(/foo/).test(s)” and of course the script still works fine if loaded by itself and now it doesn’t raise an exception in jsmin.rb. Though I still can’t figure out why it’s not working properly. Every other script works fine, but tablesorter 2.0 just fails to work, with no errors (that I can see with firebug anyway). It’s one of those things that seems nearly impossible to troubleshoot. sigh

Anybody have any suggestions for how to troubleshoot such a problem? I can’t tell if this is a jsmin problem or poorly written js being passed in.

It seems the minify_cache (posted by Jason on 3/16/09) is now part of Rails core. So the merging aspect of this plugin is no longer necessary.

I set up a cap task to run yuicompressor on each *.js and *.css file, then use javascript_include_tag :cache => true to make Rails combine them into one.