tag:blogger.com,1999:blog-27363272278830264352024-02-20T17:47:26.749+08:00It Takes a Whole Life to be ArrixArrix writes about web development, programming and all his weird thoughtsArrixhttp://www.blogger.com/profile/11483857651210475107noreply@blogger.comBlogger15125tag:blogger.com,1999:blog-2736327227883026435.post-44111213016191973632010-11-20T01:11:00.004+08:002010-11-23T10:49:12.978+08:00Server side readability with node.js<h3>For people just looking for code: <a href="https://github.com/arrix/node-readability">node-readability on github</a></h3><p><a href="http://lab.arc90.com/experiments/readability/">Readability by Arc90</a> is a fantastic javascript tool that makes web pages easier and more enjoyable to read. It removes the clutter around the article content and applies legible and beautiful styles. Apple has <a href="http://www.theregister.co.uk/2010/06/08/safari_reader_based_on_open_source_project/">incorporated</a> it into <a href="http://www.apple.com/safari/whats-new.html#reader">Safari Reader</a>. Many other apps have integrated readability, too.</p><p>While it's fast and pleasant on a modern desktop browser, the performance on resource limited devices is still unsatisfactory. It often takes several seconds to process a page on my iPad. On Nexus One, the time is much longer.</p><p>Several efforts have been made to port the readability algorithm to server side. Including:<br />
<ul><li>Ruby <a href="https://github.com/iterationlabs/ruby-readability">ruby-readability</a></li>
<li>Python <a href="https://github.com/gfxmonk/python-readability">python-readability</a> <a href="http://nirmalpatel.com/fcgi/hn.py">hn.py</a> </li>
<li>C# <a href="http://code.google.com/p/nreadability/">nreadability</a> </li>
<li>PHP <a href="http://www.keyvan.net/2010/08/php-readability/">php-readability</a> </li>
</ul>All these ports have deviated from the original implementation to adapt to the target technology, which means they may produce different result and extra work must be done to keep them up to date if a newer version of readability comes out.</p><p>It would be nice if we could run readability.js in a server side javascript host environment so that<br />
<ul><li>The result is as close to that in browsers as possible</li>
<li>Minimal changes are required</li>
</ul></p><p>So I took a stab to adapt readability.js to server side using <a href="http://nodejs.org" title="Evented I/O for V8 JavaScript">node.js</a> and <a href="https://github.com/tmpvar/jsdom" title="CommonJS implementation of the DOM intended to be platform independent and as minimal/light as possible while completely adhering to the w3c DOM specifications.">jsdom</a>. The code is available on <a href="https://github.com/arrix/node-readability">github</a> and there is a live demo (coming soon). The result is quite good in my testing except it's a bit slow.</p><p>Here is an example usage<br />
<pre class="brush: js">var fs = require('fs'),
readability = require('./lib/readability.js');
var html = fs.readFileSync('test/nytime.html', 'utf-8');
// This is an very early example. The API is subject to change.
readability.parse(html, 'http://www.example.com/somepage.html', function(result) {
console.log(result.title, result.content);
});
</pre></p><h3>Porting readability.js to node.js</h3><p>There isn't a full browser environment available for node.js.<br />
Features that can not easily made to work are disabled for now. e.g. Fetching next pages, iframe support.</p><p>Another reason for disabling the two features is to keep the code synchronized. node.js is a single threaded event driven execution environment. There is nothing like locking. But readability is written as a singleton one shot object. So I have to reset states before before every run. If the code stops halfway to wait for IO, it might be re-entered before the current run finishes, which surely must be avoided.</p><p>Some code paths are disabled because they don't make sense in a non-browser environment but may cause problems.</p><p>Some NodeList iteration loops are slightly modified to work around a <a href="https://github.com/tmpvar/jsdom/issues/#issue/77">jsdom limitation</a> where a live NodeList isn't updated automatically after DOM changes when accessed via indexing.</p><p>Readability looks for comma(, ) when calculating scores. I've extracted hard coded literals to a variable so that it can be configured to match punctuations in multiple languages. <code>var reComma = /[\uff0c,]/; //chinese comma, too</code></p><p>Readability UI elements, header and footer aren't included in the result. This is merely done to allow more flexible usage. I'd like to include <a href="http://twitter.com/chrisdary/status/15672452287">acknowledgement to Arc90</a> in any final product and suggest all of you do the same.</p><p>Most time is spent for performance optimization. See below.</p><h3>Performance</h3><p>The first working version was incredibly slow. It was common to take 5-10 seconds to process a moderately sized page. Certain pages can take minutes as if the process is freezing. While node.js uses the very fast <a href="http://code.google.com/p/v8/">V8 javascript engine</a>, the DOM implemented in jsdom uses pure javascript and isn't optimized for performance yet.</p><p>I added simple profiling code so that I can see how much time is taken by each step and find code paths worth optimizing most. Below is a sample output for <a href="http://en.wikipedia.org/wiki/Ruby">http://en.wikipedia.org/wiki/Ruby</a></p><pre>19 Nov 20:57:32 - ---DOM created
19 Nov 20:57:32 - 0 seconds [Remove all stylesheets]
19 Nov 20:57:32 - 0 seconds [Turn all double br's into p's]
19 Nov 20:57:32 - 0.05 seconds [prepDocument]
19 Nov 20:57:33 - 0.455 seconds [grabArticle nodePrepping]
19 Nov 20:57:33 - 0.015 seconds [grabArticle calculate scores]
19 Nov 20:57:33 - 0.227 seconds [grabArticle find top candidate]
19 Nov 20:57:33 - 0.033 seconds [grabArticle look through its siblings]
19 Nov 20:57:34 - 0.043 seconds [cleanConditionally]
19 Nov 20:57:34 - 0.2 seconds [cleanConditionally]
19 Nov 20:57:34 - 0.032 seconds [cleanConditionally]
19 Nov 20:57:34 - 0.054 seconds [cleanConditionally]
19 Nov 20:57:34 - 0.026 seconds [prepArticle Remove extra paragraphs]
19 Nov 20:57:34 - 0.206 seconds [prepArticle innerHTML replacement]
19 Nov 20:57:34 - 1.372 seconds [prepArticle]
19 Nov 20:57:34 - 2.407 seconds [grabArticle]
19 Nov 20:57:34 - 2.53 seconds [================= TOTAL]
Profiling summary ==========================
1 2.530 ================= TOTAL
1 0.050 prepDocument
1 0.000 Remove all stylesheets
1 0.000 Turn all double br's into p's
1 2.407 grabArticle
1 0.455 grabArticle nodePrepping
1 0.015 grabArticle calculate scores
2338 0.071 getInnerText
1 0.227 grabArticle find top candidate
105 0.259 getLinkDensity
1 0.033 grabArticle look through its siblings
1 1.372 prepArticle
4 0.329 cleanConditionally
1 0.026 prepArticle Remove extra paragraphs
1 0.206 prepArticle innerHTML replacement</pre><p>As shown in the summary, getInnerText is called many times, that's actually one function that I made hundredfold faster, cutting the running time by seconds.<br />
element.textContent is rather slow in jsdom, so I replaced it with a tree walker.<br />
<pre class="brush:js">function TextWalker(node, func) {
function walk(cur) {
var children, len, i;
if (cur.nodeType == 3) {
func(cur);
return;
} else if (cur.nodeType != 1) {
return;
}
children = cur.childNodes;
for (i = 0, len = children.length; i < len; i++) {
walk(children[i]);
}
}
walk(node);
}
var textContent = '';
TextWalker(e, function(cur) {
textContent += cur.nodeValue;
});
</pre></p><p>Tree walkers like above are also used in other places to speed up NodeList iteration. As we know, the <code>getElementsByTagName()</code> function family return a live NodeList which is updated automatically when the DOM changes. Live NoteLists are very fast in most browsers because of highly efficient caching. That's why <a href="http://www.nczonline.net/blog/2010/09/28/why-is-getelementsbytagname-faster-that-queryselectorall/" title="Why is getElementsByTagName() faster that querySelectorAll()?"><code>getElementsByTagName()</code> is much faster than <code>querySelectorAll()</code></a>.</p><p>But in jsdom, things are quite opposite. Keeping live NodeLists up to date is very expensive in jsdom <del>because there is no caching at all. In a tight loop that modifies DOM, live NodeLists are just unaffordable. </del><ins><b>Update:</b> There is <a href="http://github.com/tmpvar/jsdom/commit/c90ad35372319bf8bf213daae4960f6ef5c04192">simple version number based caching</a> now but a tree walker is still much faster.</ins></p><p>So a carefully crafted tree walker is used to replace live NodeList in the "node prepping" part in <code>grabArticle()</code>. This optimization is significant, reducing running time for certain pages from several minutes to seconds.</p><h4>So far so fast - 1.1 seconds per page</h4><p>These optimizations turned out to be very effective. In my testing of 140 pages with an average size of <b>58KB</b> collected from <a href="http://digg.com/news.rss" title="Digg Top News">digg</a>, <a href="http://feeds.delicious.com/v2/rss/?count=50" title="Delicious Hot List">delicious</a> and <a href="http://news.ycombinator.com/rss" title="Hacker News">hacker news</a>, the average time taken for each page is about <b>1.1 seconds</b> on a Mac Mini (2.4G Intel Core 2 Duo). </p><p>The task is CPU bound. The running time is often not linear to DOM size. DOM content and structure can greatly impact performance. The slowest case is when readability fails to extract enough content so it reruns the algorithm with more aggressive configurations. I believe that with more specific tuning for jsdom, the running time can be further reduced.</p><h3>Limitations</h3><p>While the port gives good result for most pages, Node.js + jsdom isn't a full browser environment so the missing features impose limitations.</p><p>Server side DOM environment doesn't understand CSS. It is reasonable for readability to make use of CSS information. For example, it could discard invisible or too small elements. Although currently readability hardly uses any CSS information, this would be an important limitation in future.</p><h3>The next step</h3><p>Coming from the world of servers and back end systems, I care about performance most. I'm going study jsdom more closely to understand its performance characters better. For now, some options in my mind include <br />
<ul><li>Replace live NodeList iteration with DOM tree transversal when applicable.</li>
<li>Combine DOM transversals. Do several things in one go.</li>
<li>Avoid large innerHTML assignment when possible. HTML parsing and DOM creation are expensive.</li>
</ul></p><p>Regarding readability.js itself. I'd like to suggest a few improvements. I'd be happy if I can contribute.<br />
<ol><li>Separate the library part from the bookmarklet part. The core algorithms can be extracted as a library. This allows the core function to be used with different front ends. e.g. A browser extension, a widget.<br />
</li>
<li>Organize the core into several modules and break big functions into smaller ones. </li>
<li>Add some hooks and more configurations so that it's possible to do page specific optimization in a plug-in manner.</li>
<li>Currently, readability is a singleton one shot object. It'll be nice to make it classy so that state management will be easier in an event driven architecture.</li>
<li>Be unobtrusive. It would be nice if the original DOM can be left intact.</li>
</ol></p><p>At the end of this post, I'd like to thank <a href="http://arc90.com/" title="Arc90 | Web Application Design & Development">Arc90</a> for building such a wonderful tool and <a href="http://github.com/tmpvar">Elijah Insua</a> (<a href="http://twitter.com/tmpvar">@tmpvar</a>), whose <a href="http://github.com/tmpvar/jsdom">jsdom</a> has opened many seriously cool possibilities for server side javascipt.</p>Arrixhttp://www.blogger.com/profile/11483857651210475107noreply@blogger.com5tag:blogger.com,1999:blog-2736327227883026435.post-1166853767055136742010-04-21T23:11:00.000+08:002010-04-21T23:11:32.612+08:00Augmented Reality on iPhone: Marker Tracking DemoI've been working on an augmented reality game app for iPhone recently. The player will be able to interact with the game by placing and moving specially designed marker cards in front of the iPhone's camera. The app analyzes video frames in real time and updates the scene accordingly. Here is demo of what I've got so far - marker tracking on iPhone 3GS. <br />
<object height="385" width="480"><param name="movie" value="http://www.youtube.com/v/Xtfn8PMUnX0&hl=en_US&fs=1&"></param><param name="allowFullScreen" value="true"></param><param name="allowscriptaccess" value="always"></param><embed src="http://www.youtube.com/v/Xtfn8PMUnX0&hl=en_US&fs=1&" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="480" height="385"></embed></object><br />
<br />
The marker tracking algorithm is done by a slightedly modified version of <a href="http://nyatla.jp/nyartoolkit/wiki/index.php?NyARToolkitCPP">NyARtoolkitCPP</a>. Analyzing each frame takes < 55ms in average. The OpenGL ES rendering can achieve 30 frames per second. The current algorithm relies on thresholding so it is very sensitive to changing lighting condition. Also, template matching based identification requires matching against the whole pattern library for each detected marker. I'm seeking for better algorithms that use edge detecting and id recognition. <a href="http://handheldar.net/stbtracker.php">StbTracker</a>, the successor of <a href="http://studierstube.icg.tu-graz.ac.at/handheld_ar/artoolkitplus.php">ARToolkitPlus</a>, looks ideal for devices such as iPhone but unfortunately it's not open source.<br />
<br />
The most discussed topic by iPhone AR application developers today is real time video frame grabbing. There isn't an elegant way yet. For now, I'm using private APIs to repeatedly fetch the camera preview view. The good news is that we are going to have<a href="http://live.gdgt.com/2010/04/08/live-iphone-os-4-0-event-coverage/"> full access to still and video camera data</a> in iPhone OS 4. A peek at the related APIs in the iPhone SDK 4 beta 2 further confirms that. <br />
<div class="separator" style="clear: both; text-align: center;"><a href="http://c1291262.cdn.cloudfiles.rackspacecloud.com/apple-iphone-os4_059.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://c1291262.cdn.cloudfiles.rackspacecloud.com/apple-iphone-os4_059.jpg" /></a></div>The OpenGL ES overlay content is displayed in a second UIWindow above the one containing the camera preview. This only point of dosing so is to get a clean preview image without the overlaying content. These ugly hacks will be all gone once we have iPhone OS 4.Arrixhttp://www.blogger.com/profile/11483857651210475107noreply@blogger.com6tag:blogger.com,1999:blog-2736327227883026435.post-36390066336390953652009-02-21T11:11:00.005+08:002009-02-21T11:39:32.569+08:00Bypassing code signing to test your app on the iPhone device<p>Apple requires apps to be signed with a signing identity before you run them on device or distributing them to App Store. To get a valid signing identity (code signing certificate + private key), you must apply for Apple's iPhone developer program and pay them $99.</p><p>It is possible to run and test your app on your jailbroken iPhone without the need to pay (you'll still have to pay if you wish to publish your app on the App Store anyway). This process is often referred to as bypassing code signing.</p><p>Open Cydia on your iPhone, in the home page, scroll down and tap Bypassing Code Signature in the Developer section, you'll see instructions to bypass code signing. There are 3 options. Option #1 is what I tried. You can find detailed instructions for the other options on the internet.<br /><br />Option #1: The verification check in the iPhone OS has been hacked so you can sign your code with any self-signing certificate and iPhone will let you pass. If you want the most integrated and smooth development experience with Xcode (Build and Go and debug on the device), this is the option for you.<br />You need to do two things.<br />The first is to bypass the Xcode verification.<br /></p><ol><li>Create a self signing certificate named iPhone Developer following the instructions by Apple <a title="Obtaning a Signing Identity" href="http://developer.apple.com/documentation/Security/Conceptual/CodeSigningGuide/Procedures/chapter_3_section_2.html" id="xz1r">Obtaning a Signing Identity</a> .<br /></li><li>in Xcode - > project - > edit project setting -> Add User - Defined Setting<br />PROVISIONING_PROFILE_ALLOWED : NO<br />PROVISIONING_PROFILE_REQUIRED : NO<br />(there seems to be a way in which you don't have to add these settings on a per project basis but I haven't tried. see <a href="http://iphone.cazisoft.com/?p=635">Developing Application for iPhone OS</a>)<span style="font-size:85%;"><br /></span></li><li>add in info.plist - > SignerIdentity : Apple iPhone OS Application Signing<br />(from <a title="http://forums.macrumors.com/showthread.php?t=640341" href="http://forums.macrumors.com/showthread.php?t=640341" id="jnwi">http://forums.macrumors.com/showthread.php?t=640341</a> )<span style="font-size:85%;"><br /></span></li></ol>Now you should be able to choose device targets and build your app. Try clean if you see code signing related errors.<br />You'll probably get a device verification error if your iPhone's Mobile Installation files haven't been patched yet. So the second thing to do is to compromise the verification on the iPhone.<br /><ol><li>add http://cydia.hackulo.us to your Cydia sources</li><li>install miPatch in Cydia</li><li>reboot your iPhone (I didn't reboot and it worked IIRC)</li></ol>Done.<br /><br />see <a href="http://iphone.cazisoft.com/?p=635">Developing Application for iPhone OS</a><span> for more information.</span><br /><br />Option #2: Just build your app and scp it to your iPhone and run ldid -S yourapp. ldid (Link Identity Editor ) is a command line package available in cydia. see <a title="http://www.ipodtouchfans.com/forums/showthread.php?t=104884" href="http://www.ipodtouchfans.com/forums/showthread.php?t=104884" id="n20r">http://www.ipodtouchfans.com/forums/showthread.php?t=104884</a><br /><br />Now I can debug on my device live - tap a button and stop at a breakpoint set in Xcode. I wouldn't want to write any iPhone app without a real Mac!<br /><br /><span style="font-weight: bold;">Legal Notice:</span> I don't live in a country where this kind of notice is favoured. But for completeness, please know that bypassing Apple's restrictions is usually illegal. However, $99 is too much for a curious guy who just wants to test his Hello World.<br /><p><span> </span></p>Arrixhttp://www.blogger.com/profile/11483857651210475107noreply@blogger.com3tag:blogger.com,1999:blog-2736327227883026435.post-80985823987118489432009-02-17T20:30:00.006+08:002009-02-17T21:18:40.123+08:00Private methods in Objective-CPrivate method in Objective-C<br /><br />Objective-C doesn't have the concept of private methods although you can use @private, @public, @protected to specify visibility scope for variables of a class.<br />One way to simulate private methods is to use category.<br /><br /><pre class="brush:cpp"><br />// in MyClass.m, before the main @implementation block<br />@interface MyClass (Private)<br />- (void)privateMethod;<br />@end<br /><br />// you won't get warnings if this block is missing<br />@implementation MyClass (Private)<br />- (void)privateMethod {<br /> //do sth...<br />}<br />@end<br /></pre><br /><br />Note that it's a good practice to write an @implementation block for the category as soon as you finish the category @interface declaration. If the @implementation block for the category is missing, the compiler won't warn you about any missing method implementations.<br /><br /><pre class="brush:cpp"><br />// in MyClass.m, before the main @implementation block<br />@interface MyClass ()<br />- (void)privateMethod;<br />@end<br /><br />@implementation MyClass<br />- (void)privateMethod {<br /> //do sth...<br />}<br />@end<br /><br /></pre><br />You can also leave the category name blank to use an <a href="http://developer.apple.com/documentation/Cocoa/Conceptual/ObjectiveC/Articles/chapter_6_section_5.html#//apple_ref/doc/uid/TP30001163-CH20-SW2">extension</a>. If you do so, you must implement the methods in the main @implementation block. The compiler will always issue warnings when implementations for the methods are missing.<br /><br />References<br /><ol><li><a href="http://developer.apple.com/documentation/Cocoa/Conceptual/ObjectiveC/Articles/chapter_6_section_1.html#//apple_ref/doc/uid/TP30001163-CH20-SW1">http://developer.apple.com/documentation/Cocoa/Conceptual/ObjectiveC/Articles/chapter_6_section_1.html#//apple_ref/doc/uid/TP30001163-CH20-SW1</a></li><li><a href="http://www.otierney.net/objective-c.html#categories">http://www.otierney.net/objective-c.html#categories</a></li><li><a href="http://stackoverflow.com/questions/172598/best-way-to-define-private-methods-for-a-class-in-objective-c">Best way to define private methods for a class in Objective-C</a><br /></li></ol>Arrixhttp://www.blogger.com/profile/11483857651210475107noreply@blogger.com0tag:blogger.com,1999:blog-2736327227883026435.post-60128352883692272592009-01-27T02:48:00.003+08:002009-02-17T21:06:30.461+08:00marquee generates swarms of scroll events in Webkit<p>The non-standard marquee tag, first introduced in early versions of IE, can be useful to create scrolling effects. Though deprecated, all major browsers today seem to support it.</p><br /><marquee>sample scrolling text in marquee</marquee><br /><br /><p>One thing you should be aware of is that in Webkit (used by Appla Safari and Google Chrome), marquee generates a steady stream of scroll events. If you have listeners to the scroll event on the document, watch out for these events which bubble up to the document! You may want to filter out them by checking event.target. For example:</p><br /><pre class="brush:js"><br />$(document).bind('scroll', function(event) {<br /> //filter out scroll events of marquees in Webkit<br /> if (event.target != document) return;<br /> //documrent scroll...<br />});<br /></pre>Arrixhttp://www.blogger.com/profile/11483857651210475107noreply@blogger.com1tag:blogger.com,1999:blog-2736327227883026435.post-26656038895738998352007-08-27T23:10:00.001+08:002007-08-28T11:33:54.086+08:00Screen scraping with jQuery<p>A test case in my work requires a complete list of HTML elements and a list of self-closing elements (e.g. <br/>).</p> <p>The <a href="http://www.w3.org/TR/html401/index/elements.html">W3C Index of HTML 4 Elements</a> lists all defined elements in a table. For each row with an "E" in the <em>Empty</em> column, the corresponding element doesn't need a closing tag (and thus is self-closing). </p> <p>With two lines of jQuery code in the Firebug console, I got the lists I wanted. Here is how:</p> <p>To get all elements</p><pre>$.map($('table tr:gt(0) a'), <span style="color: #0000ff">function</span>(e) {<span style="color: #0000ff">return</span> $.trim($(e).text());})</pre><br /><p>To get all self-closing elements (formatted for readability)</p><pre>$.map(<br> $('table tr:gt(0)').filter(<span style="color: #0000ff">function</span>() {<br> <span style="color: #0000ff">return</span> $(<span style="color: #0000ff">this</span>).<span style="color: #0000ff">find</span>('td:nth-child(4)').text() == 'E';<br> }), <br> <span style="color: #0000ff">function</span>(e){<span style="color: #0000ff">return</span> $.trim($(e).<span style="color: #0000ff">find</span>('td:first-child').text());});</pre><pre>$.trim() is needed because the HTML source contains \n in the <em>Name</em> column.</pre><pre>This demonstrates a handy usage of jQuery as a hacking tool. Another excellent demonstration can be found <a title="Hacking Digg With Firebug and jQuery" href="http://ejohn.org/blog/hacking-digg-with-firebug-and-jquery/">here</a>.</pre><pre>You can add jQuery to the current page using the <a href="javascript:var%20s=document.createElement('script');s.setAttribute('src',%20'http://code.jquery.com/jquery-nightly.js');document.getElementsByTagName('body')[0].appendChild(s);alert('thank you for using jquery!');void(s);">jQuerify</a> bookmarklet.</pre><pre>Happy jQuerifying!<br></pre>Arrixhttp://www.blogger.com/profile/11483857651210475107noreply@blogger.com0tag:blogger.com,1999:blog-2736327227883026435.post-82591566105088604792007-07-26T13:15:00.001+08:002007-07-26T13:15:49.187+08:00Ruby code: Finding the closest point<p>We want to fetch an avatar of specific size. How to find the closest size from all available sizes? Suppose available sizes are 16, 48 and 96.</p> <p>An obvious version is</p><pre> <span style="color: #0000ff">def</span> closest_size(size)<br /> <span style="color: #0000ff">case</span><br /> <span style="color: #0000ff">when</span> size < 32<br /> 16<br /> <span style="color: #0000ff">when</span> size < 72<br /> 48<br /> <span style="color: #0000ff">else</span><br /> 96<br /> <span style="color: #0000ff">end</span><br /> <span style="color: #0000ff">end</span></pre><br /><p>The value used in comparison tests is the average of two neighboring candidate sizes. The problem is that when our available sizes change, we need to add or remove when clauses and recalculate average values.</p><br /><p>Here is a smarter way</p><pre><span style="color: #0000ff">def</span> closest_size(size)<br /> points = [16, 48, 96]<br /> distances = points.map {|a| (size - a).abs}<br /> points[distances.index(distances.min)]<br /><span style="color: #0000ff">end</span></pre><br /><p>It finds the point with shortest distance to the given size. Now if we want to change candidate sizes, we only need to change the array literal. Further, we can even pass the candidate array as an argument.</p>Arrixhttp://www.blogger.com/profile/11483857651210475107noreply@blogger.com4tag:blogger.com,1999:blog-2736327227883026435.post-12861261182521948322007-07-26T11:46:00.001+08:002007-07-26T12:10:01.937+08:00Rails: Error calling Dispatcher.dispatch...<p>I ran into a strange issue with one of my rails projects today - the browser displayed html source code in plain text instead of rendering. Nothing is listed in Firebug's net tab. Strange! Later I found the server was always sending back HTTP 404 with Content-type: text/plain. In the log, there were many lines of "Error calling Dispatcher.dispatch #<NameError: cannot remove Object::Handler>".</p> <p>Finally, I found out that in a controller, someone has put <em><em>include</em> xxx</em> at file level and in that module he defined class Handler. So that was it. After moving <em><em><em>include</em> </em>xxx</em> into the controller class definition, everything went well.</p> <p>File level <em>include</em> (<em>include</em> outside class/module definition) actually affects <em>Object</em> - the mother of all ruby elves. So it's something we should definitely avoid in real world projects.</p> <p><strong>Never <em>include</em> outside <em>class/module</em> definition.</strong></p> <p>BTW, Rails is notoriously good at giving error messages unrelated to the cause of the problem. This is largely due to the dynamic nature of Ruby.</p>Arrixhttp://www.blogger.com/profile/11483857651210475107noreply@blogger.com3tag:blogger.com,1999:blog-2736327227883026435.post-45099591994724203742007-05-06T15:11:00.003+08:002008-08-18T01:13:39.524+08:00Ubuntu 7.04/8.04 and Mouse Wheel<p>I've just installed unbuntu 7.04 in VMWare. Everything is cool except that the mouse wheel stops working.</p> <p>sudo gedit /etc/X11/xorg.conf</p> <p>Find the mouse section which may look like</p> <p>Section "InputDevice"<br />Identifier "Configured Mouse"<br />Driver "mouse"<br />Option "CorePointer"<br />Option "Device" "/dev/input/mice"<br />Option "Protocol" "ps/2"<br />Option "ZAxisMapping" "4 5"<br />Option "Emulate3Buttons" "true"<br />EndSection </p><p>Change Options "Protocol" "ps/2" to Option "Protocol" "IMPS/2" </p><p>Save the file and restart X (ctrl + alt + backspace). </p><p>It works for me. Happy scrolling!</p><p><span style="font-weight: bold;">UPDATE: 2008-8-18</span></p><p>Now with Ubuntu 8.04 running in VMWare Workstation 6.04 on Windows Server 2008...</p><p>The working config for me is as below:</p><p>Section "InputDevice"<br /> Identifier "Configured Mouse"<br /> Driver "vmmouse"<br /> Option "CorePointer"<br /> Option "Device" "/dev/input/mice"<br /> Option "Protocol" "ImPS/2"<br /> Option "ZAxisMapping" "4 5"<br />EndSection</p><p>The mouse cursor moves very smoothly crosses hosting and virtual machine seamlessly.<br /></p>Arrixhttp://www.blogger.com/profile/11483857651210475107noreply@blogger.com3tag:blogger.com,1999:blog-2736327227883026435.post-223367591266650302007-04-30T16:55:00.001+08:002007-04-30T16:56:02.313+08:00eval() bug in IE<p>eval('function(){}') evaluates to undefined in IE, the same for eval('(function(){})').</p> <p>How to get the Function?</p> <p>eval('[function(){}][0]')</p> <p>IE, you always let me down!</p>Arrixhttp://www.blogger.com/profile/11483857651210475107noreply@blogger.com1tag:blogger.com,1999:blog-2736327227883026435.post-69683072184985318742007-02-10T23:19:00.001+08:002007-09-08T10:36:01.668+08:00Tricky alternatives to toString() and parseInt()<h3>JavaScript is Loosely typed</h3> <p>JavaScript is dynamically typed, making things extremely flexible. Suppose you have a text field for inputting birth year and want to greet to people born after 1984, simply write </p><pre class="code"><span style="color: rgb(0,0,200)">var</span> birthYear = <span style="color: rgb(0,100,200)">document</span>.getElementById(<span style="color: rgb(0,128,0)">'txtBirthYear'</span>).value;<br><span style="color: rgb(0,0,200)">if</span> (birthYear > 1984) {<br> <span style="color: rgb(0,100,200)">alert</span>(<span style="color: rgb(0,128,0)">'Greetings!'</span>); <br>}</pre><a href="http://11011.net/software/vspaste"></a><pre class="code"></pre><br><br /><p>When JavaScript sees you compare a String with a Number, it automatically converts the String to Number and the perform comparison.</p><br><br /><p>But sometimes the ambiguity of type causes problems. 1 + '1' evaluates to '11' instead of 2. This may cause hard to find bugs. <a href="http://www.crockford.com/">Douglas Crockford</a> categorizes "the overloading of + to mean both addition and concatenation with type coercion" as a design error of JavaScript in his famous article on <a href="http://javascript.crockford.com/javascript.html">The World's Most Misunderstood Programming Language</a>.</p><br><br /><p>Still we often need to convert data type between String and Number. To convert variable i to String, simply call i.toString(). To convert s to a Number, we use Number(s). This is nice and clear. </p><br><br /><h3>The empty string ('') concatenation trick, the plus sign (+) trick and the minus zero (- 0) trick</h3><br><br /><p>But for guys who want to squeeze every byte. There are tricky alternatives. <br><br /><ul><br><br /><li>To convert x to String: x + '' <br /><li>To convert x to Number: +x<br /><li>To convert x to Number: x - 0</li></ul><br><br /><p></p><br><br /><p>For examples, </p><pre class="code">1 + 2 + 3 <span style="color: rgb(128,128,0)">//produces</span> <span style="color: rgb(128,128,0)">6<br></span><span style="color: rgb(128,128,0)">//while<br></span><span style="color: rgb(0,128,0)">'1'</span> + 2 + 3 <span style="color: rgb(128,128,0)">//produces</span> <span style="color: rgb(128,128,0)">'123'</span></pre><span style="color: rgb(0,128,0)">'1'</span> - 0 + 2 + 3 <span style="color: rgb(128,128,0)">//produces</span> <font color="#808000">6<span style="color: rgb(128,128,0)"></span></font><span style="color: rgb(128,128,0)"><br></span><span style="color: rgb(0,128,0)">'1'</span> + <span style="color: rgb(0,128,0)">'2'</span> <span style="color: rgb(128,128,0)">//produces</span> <span style="color: rgb(128,128,0)">'12'<br></span><span style="color: rgb(128,128,0)">//while<br></span>+<span style="color: rgb(0,128,0)">'1'</span> + +<span style="color: rgb(0,128,0)">'2'</span> <span style="color: rgb(128,128,0)">//produces</span> <font color="#808000">3</font><a href="http://11011.net/software/vspaste"></a><br><br /><p>Notice that +x and x-0 doesn't mean parseInt(x) or parseFloat(x), it doesn't do any further parsing.</p><pre class="code"><span style="color: rgb(0,100,200)">parseInt</span>(<span style="color: rgb(0,128,0)">'2007 is promising'</span>) <span style="color: rgb(128,128,0)">//produces</span> <span style="color: rgb(128,128,0)">2007<br></span><span style="color: rgb(128,128,0)">//while<br></span>+<span style="color: rgb(0,128,0)">'2007 is promising'</span> <span style="color: rgb(128,128,0)">//produces</span> <span style="color: rgb(128,128,0)">NaN</span></pre><pre class="code"><span style="color: rgb(128,128,0)"><font face="Verdana" color="#333333">Let's call them <em>the empty string concatenation</em> conversion trick, <em>the plus sign</em> and <em>the minus zero</em> trick. Both of the tricks sacrifice clarity and make code harder to understand.</font></span></pre>Arrixhttp://www.blogger.com/profile/11483857651210475107noreply@blogger.com3tag:blogger.com,1999:blog-2736327227883026435.post-11589385943332241692007-02-06T17:06:00.001+08:002007-02-11T13:25:48.337+08:00Closure, eval and Function<p>eval() evaluates a string of JavaScript code. The Function constructor can be used to create a function from string. Someone says that the Function constructor is another form of eval(). However, one significant difference between eval() and the Function constructor is that while eval() keeps the lexical scope, the Function constructor always creates top-level functions.</p><pre class="code"><span style="color: rgb(0,0,200)">function</span> f1() {<br /> <span style="color: rgb(0,0,200)">var</span> bbb = 2; <br /> <span style="color: rgb(0,100,200)">eval</span>(<span style="color: rgb(0,128,0)">'alert(bbb);'</span>);<br />}<br />f1(); <span style="color: rgb(128,128,0)">//alerts</span> <span style="color: rgb(128,128,0)">2<br /><br /></span><span style="color: rgb(0,0,200)">function</span> f2() {<br /> <span style="color: rgb(0,0,200)">var</span> bbb = 2; <br /> <span style="color: rgb(0,0,200)">new</span> <span style="color: rgb(0,100,200)">Function</span>(<span style="color: rgb(0,128,0)">'alert(bbb)'</span>)();<br />} <br />f2(); <span style="color: rgb(128,128,0)">//bbb</span> <span style="color: rgb(128,128,0)">undefined</span> <span style="color: rgb(128,128,0)">error<br /><br /></span><span style="color: rgb(0,0,200)">function</span> f3() {<br /> <span style="color: rgb(0,0,200)">var</span> bbb = 2; <br /> <span style="color: rgb(0,100,200)">eval</span>(<span style="color: rgb(0,128,0)">'function() {alert(bbb);}'</span>)();<br />}<br />f3(); <span style="color: rgb(128,128,0)">//alerts</span> <span style="color: rgb(128,128,0)">2</span></pre><a href="http://11011.net/software/vspaste"></a><br /><p>eval() inside a function body creates a closure while new Function() doesn't. This difference may not bother you for the whole lifetime. However, it happens to bother me once. It's about <a href="http://jquery.com">jQuery</a> - a new type of JavaScript library. I'm using jQuery in my bookmarklet application. In order to make the code as unobtrusive as possible, I decided to put all my code including the jQuery code inside an anonymous function. It looks like this:<pre class="code">(<span style="color: rgb(0,0,200)">function</span>() {<br /> <span style="color: rgb(128,128,0)">//jQuery</span> <span style="color: rgb(128,128,0)">code<br /></span> <span style="color: rgb(128,128,0)">//my</span> <span style="color: rgb(128,128,0)">code<br /></span>})();</pre><a href="http://11011.net/software/vspaste"></a><br /><p>In this way, even the jQuery object is just a local variable. The outside environment is completely unaffected. But $().parents(), $().children, $().prev(), $().next() and $().siblings() always fail in my code. These functions are created by the Function constructor in $.grep() and $.map(). </p><pre class="code"><span style="color: rgb(128,128,0)">//</span> <span style="color: rgb(128,128,0)">If</span> <span style="color: rgb(128,128,0)">a</span> <span style="color: rgb(128,128,0)">string</span> <span style="color: rgb(128,128,0)">is</span> <span style="color: rgb(128,128,0)">passed</span> <span style="color: rgb(128,128,0)">in</span> <span style="color: rgb(128,128,0)">for</span> <span style="color: rgb(128,128,0)">the</span> <span style="color: rgb(128,128,0)">function,</span> <span style="color: rgb(128,128,0)">make</span> <span style="color: rgb(128,128,0)">a</span> <span style="color: rgb(128,128,0)">function<br /></span><span style="color: rgb(128,128,0)">//</span> <span style="color: rgb(128,128,0)">for</span> <span style="color: rgb(128,128,0)">it</span> <span style="color: rgb(128,128,0)">(a</span> <span style="color: rgb(128,128,0)">handy</span> <span style="color: rgb(128,128,0)">shortcut)<br /></span><span style="color: rgb(0,0,200)">if</span> ( <span style="color: rgb(0,0,200)">typeof</span> fn == <span style="color: rgb(0,128,0)">"string"</span> )<br /> fn = <span style="color: rgb(0,0,200)">new</span> <span style="color: rgb(0,100,200)">Function</span>(<span style="color: rgb(0,128,0)">"a"</span>,<span style="color: rgb(0,128,0)">"i"</span>,<span style="color: rgb(0,128,0)">"return "</span> + fn);</pre><a href="http://11011.net/software/vspaste"></a><br /><p>So they are all top-level and the identifier "jQuery" inside is resolved as window.jQuery which is undefined and the code fails.</p><br /><p>We can implement an alternative to the Function constructor and use it within the lexical scope:</p><pre class="code"><span style="color: rgb(0,0,200)">var</span> createFunc = (<span style="color: rgb(0,0,200)">function</span> () {<br /> <span style="color: rgb(0,0,200)">var</span> args = [].slice.call(<span style="color: rgb(0,100,200)">arguments</span>);<br /> <span style="color: rgb(0,0,200)">var</span> body = args.pop();<br /> <span style="color: rgb(0,0,200)">return</span> <span style="color: rgb(0,100,200)">eval</span>(<span style="color: rgb(0,128,0)">'function('</span> + args.join(<span style="color: rgb(0,128,0)">','</span>) + <span style="color: rgb(0,128,0)">') {'</span> + body + <span style="color: rgb(0,128,0)">'}'</span>);<br />}).<span style="color: rgb(0,100,200)">toString</span>();<br /><br /><span style="color: rgb(0,0,200)">function</span> f4() {<br /> <span style="color: rgb(0,0,200)">var</span> bbb = 2;<br /> <span style="color: rgb(0,100,200)">eval</span>(createFunc)(<span style="color: rgb(0,128,0)">'alert(bbb);'</span>)(); <br />}<br />f4(); //alerts 2</pre><pre class="code">You can use eval(createFunc) just like new Function(), but you get the bonus lexical scope binding.</pre><pre class="code"><span style="color: rgb(0,0,200)">function</span> f6() {<br /> <span style="color: rgb(0,0,200)">var</span> add = <span style="color: rgb(0,0,200)">function</span>(a, b) {<span style="color: rgb(0,0,200)">return</span> a + b;};<br /> <span style="color: rgb(0,0,200)">return</span> <span style="color: rgb(0,100,200)">eval</span>(createFunc)(<span style="color: rgb(0,128,0)">'x'</span>, <span style="color: rgb(0,128,0)">'y'</span>, <span style="color: rgb(0,128,0)">'return add(x, y);'</span>);<br />}<br />f6()(3, 5); <span style="color: rgb(128,128,0)">//8</span></pre><a href="http://11011.net/software/vspaste"><a href="http://11011.net/software/vspaste"></a><br /><p>At last, I quote <a href="http://www.crockford.com/">Douglas Crockford</a>'s <a href="http://javascript.crockford.com/code.html">words</a> on eval() and the Function constructor</p><br /><p>"<code>eval</code> is Evil </p><br /><p>The <code>eval</code> function is the most misused feature of JavaScript. Avoid it. </p><br /><p><code>eval</code> has aliases. Do not use the <code>Function</code> constructor. Do not pass strings to <code>setTimeout</code> or <code>setInterval</code>. "</p>Arrixhttp://www.blogger.com/profile/11483857651210475107noreply@blogger.com4tag:blogger.com,1999:blog-2736327227883026435.post-31798301385590787122007-01-19T22:37:00.001+08:002007-02-06T17:25:18.997+08:00Several Controversial Points in Pro JavaScript Techniques<p>I'm previewing the <a title="Pro JavaScript Techniques" href="http://jspro.org/">the ultimate JavaScript book for the modern web developer</a>. It's a great book. I strongly recommend you read it and I'm sure that you'll thank me. To make it even better, I'd like to point out and discuss several controversial points.</p> <h3>A side effect of the anonymous function scope induction trick</h3> <p>At the end of Chapter 2 >> Privileged Methods</p> <p>Listing 2-25. Example of Dynamically Generated Methods That Are Created When a New Object Is Instantiated</p><pre><span style="color: #008000">// Create a new user object that accepts an object of properties</span><br /><span style="color: #0000ff">function</span> User( properties ) {<br /> <span style="color: #008000">// Iterate through the properties of the object, and make sure</span><br /> <span style="color: #008000">// that it's properly scoped (as discussed previously)</span><br /> <span style="color: #0000ff">for</span> ( <span style="color: #0000ff">var</span> i <span style="color: #0000ff">in</span> properties ) { (<span style="color: #0000ff">function</span>(){<br /> <span style="color: #008000">// Create a new getter for the property</span><br /> <span style="color: #0000ff">this</span>[ "<span style="color: #8b0000">get</span>" + i ] = <span style="color: #0000ff">function</span>() {<br /> <span style="color: #0000ff">return</span> properties[i];<br /> };<br /> <span style="color: #008000">// Create a new setter for the property</span><br /> <span style="color: #0000ff">this</span>[ "<span style="color: #8b0000">set</span>" + i ] = <span style="color: #0000ff">function</span>(val) {<br /> properties[i] = val;<br /> };<br /> })(); }<br />}<br /><span style="color: #008000">// Create a new user object instance and pass in an object of</span><br /><span style="color: #008000">// properties to seed it with</span><br /><span style="color: #0000ff">var</span> user = <span style="color: #0000ff">new</span> User({<br /> <span style="color: #0000ff">name</span>: "<span style="color: #8b0000">Bob</span>",<br /> age: 44<br />});<br /><span style="color: #008000">// Just note that the name property does not exist, as it's private</span><br /><span style="color: #008000">// within the properties object</span><br /><span style="color: #0000ff">alert</span>( user.<span style="color: #0000ff">name</span> == <span style="color: #0000ff">null</span> );<br /><span style="color: #008000">// However, we're able to access its value using the new getname()</span><br /><span style="color: #008000">// method, that was dynamically generated</span><br /><span style="color: #0000ff">alert</span>( user.getname() == "<span style="color: #8b0000">Bob</span>" );<br /><span style="color: #008000">// Finally, we can see that it's possible to set and get the age using</span><br /><span style="color: #008000">// the newly generated functions</span><br />user.setage( 22 );<br /><span style="color: #0000ff">alert</span>( user.getage() == 22 );</pre><br /><p>The example code won't work as expected. My test with Firefox 2.0.0.1 shows that the user.getname and user.getage are actually undefined. But window.getname and window.getage are there! The error is caused by the scope induction trick: <br>(function(){})(). Inside the anonymous function, the this variable somehow points to the window object! In the simplest case:</p><pre><span style="color: #0000ff">var</span> o = {f: <span style="color: #0000ff">function</span>() {(<span style="color: #0000ff">function</span>(){<span style="color: #0000ff">alert</span>(<span style="color: #0000ff">this</span> === <span style="color: #0000ff">window</span>);})();}}; o.f(); <br><span style="color: #008000">//alerts true (but false if you evaluate the whole line in Firebug 1.0b8)</span></pre><br /><p><span style="color: #008000"></span>Seems that the implementation treats anonymous functions as properties of the window object?</p><br /><h3>null, 0, ‘’, false, and undefined are NOT all equal (==) to each other</h3><br /><p>In Chapter 3 >> != and == vs. !== and ===</p><br /><p>"...In JavaScript, null, 0, ‘’, false, and undefined are all equal (==) to each other, since they all evaluate to false... " <br /><p>Listing 3-12. Examples of How != and == Differ from !== and ===<br>// Both of these are true<br>null == false<br>0 == undefined<br>// You should use !== or === instead<br>null !== false<br>false === false</p><br /><p>Actually 0, '' and false all equal (==) to each other and null equals (==) to undefined but both null == false and undefined == false evaluate to false. This is reasonable because both null and undefined indicate "no value" while false is a valid value.</p><br /><h3>domReady Race Conditions</h3><br /><p>In Chapter 5 >> Figuring Out When the DOM Is Loaded<br>Listing 5-12. A Function for Watching the DOM Until It’s Ready</p><pre class="code"><span style="color: rgb(0,0,200)">function</span> domReady( f ) {<br /> <span style="color: rgb(128,128,0)">//</span> <span style="color: rgb(128,128,0)">If</span> <span style="color: rgb(128,128,0)">the</span> <span style="color: rgb(128,128,0)">DOM</span> <span style="color: rgb(128,128,0)">is</span> <span style="color: rgb(128,128,0)">already</span> <span style="color: rgb(128,128,0)">loaded,</span> <span style="color: rgb(128,128,0)">execute</span> <span style="color: rgb(128,128,0)">the</span> <span style="color: rgb(128,128,0)">function</span> <span style="color: rgb(128,128,0)">right</span> <span style="color: rgb(128,128,0)">away<br /></span> <span style="color: rgb(0,0,200)">if</span> ( domReady.done ) <span style="color: rgb(0,0,200)">return</span> f();<br /> <span style="color: rgb(128,128,0)">//</span> <span style="color: rgb(128,128,0)">If</span> <span style="color: rgb(128,128,0)">we've</span> <span style="color: rgb(128,128,0)">already</span> <span style="color: rgb(128,128,0)">added</span> <span style="color: rgb(128,128,0)">a</span> <span style="color: rgb(128,128,0)">function<br /></span> <span style="color: rgb(0,0,200)">if</span> ( domReady.timer ) {<br /> <span style="color: rgb(128,128,0)">//</span> <span style="color: rgb(128,128,0)">Add</span> <span style="color: rgb(128,128,0)">it</span> <span style="color: rgb(128,128,0)">to</span> <span style="color: rgb(128,128,0)">the</span> <span style="color: rgb(128,128,0)">list</span> <span style="color: rgb(128,128,0)">of</span> <span style="color: rgb(128,128,0)">functions</span> <span style="color: rgb(128,128,0)">to</span> <span style="color: rgb(128,128,0)">execute<br /></span> domReady.ready.push( f );<br /> } <span style="color: rgb(0,0,200)">else</span> {<br /> <span style="color: rgb(128,128,0)">//</span> <span style="color: rgb(128,128,0)">Attach</span> <span style="color: rgb(128,128,0)">an</span> <span style="color: rgb(128,128,0)">event</span> <span style="color: rgb(128,128,0)">for</span> <span style="color: rgb(128,128,0)">when</span> <span style="color: rgb(128,128,0)">the</span> <span style="color: rgb(128,128,0)">page</span> <span style="color: rgb(128,128,0)">finishes</span> <span style="color: rgb(128,128,0)">loading,<br /></span> <span style="color: rgb(128,128,0)">//</span> <span style="color: rgb(128,128,0)">just</span> <span style="color: rgb(128,128,0)">in</span> <span style="color: rgb(128,128,0)">case</span> <span style="color: rgb(128,128,0)">it</span> <span style="color: rgb(128,128,0)">finishes</span> <span style="color: rgb(128,128,0)">first.</span> <span style="color: rgb(128,128,0)">Uses</span> <span style="color: rgb(128,128,0)">addEvent.<br /></span> addEvent( <span style="color: rgb(0,100,200)">window</span>, <span style="color: rgb(0,128,0)">"load"</span>, isDOMReady );<br /> <span style="color: rgb(128,128,0)">//</span> <span style="color: rgb(128,128,0)">Initialize</span> <span style="color: rgb(128,128,0)">the</span> <span style="color: rgb(128,128,0)">array</span> <span style="color: rgb(128,128,0)">of</span> <span style="color: rgb(128,128,0)">functions</span> <span style="color: rgb(128,128,0)">to</span> <span style="color: rgb(128,128,0)">execute<br /></span> domReady.ready = [ f ];<br /> <span style="color: rgb(128,128,0)">//</span> <span style="color: rgb(128,128,0)">Check</span> <span style="color: rgb(128,128,0)">to</span> <span style="color: rgb(128,128,0)">see</span> <span style="color: rgb(128,128,0)">if</span> <span style="color: rgb(128,128,0)">the</span> <span style="color: rgb(128,128,0)">DOM</span> <span style="color: rgb(128,128,0)">is</span> <span style="color: rgb(128,128,0)">ready</span> <span style="color: rgb(128,128,0)">as</span> <span style="color: rgb(128,128,0)">quickly</span> <span style="color: rgb(128,128,0)">as</span> <span style="color: rgb(128,128,0)">possible<br /></span> domReady.timer = <span style="color: rgb(0,100,200)">setInterval</span>( isDOMReady, 13);<br /> }<br />}<br /><br /><span style="color: rgb(128,128,0)">//</span> <span style="color: rgb(128,128,0)">Checks</span> <span style="color: rgb(128,128,0)">to</span> <span style="color: rgb(128,128,0)">see</span> <span style="color: rgb(128,128,0)">if</span> <span style="color: rgb(128,128,0)">the</span> <span style="color: rgb(128,128,0)">DOM</span> <span style="color: rgb(128,128,0)">is</span> <span style="color: rgb(128,128,0)">ready</span> <span style="color: rgb(128,128,0)">for</span> <span style="color: rgb(128,128,0)">navigation<br /></span><span style="color: rgb(0,0,200)">function</span> isDOMReady() {<br /> <span style="color: rgb(128,128,0)">//</span> <span style="color: rgb(128,128,0)">If</span> <span style="color: rgb(128,128,0)">we</span> <span style="color: rgb(128,128,0)">already</span> <span style="color: rgb(128,128,0)">figured</span> <span style="color: rgb(128,128,0)">out</span> <span style="color: rgb(128,128,0)">that</span> <span style="color: rgb(128,128,0)">the</span> <span style="color: rgb(128,128,0)">page</span> <span style="color: rgb(128,128,0)">is</span> <span style="color: rgb(128,128,0)">ready,</span> <span style="color: rgb(128,128,0)">ignore<br /></span> <span style="color: rgb(0,0,200)">if</span> ( domReady.done ) <span style="color: rgb(0,0,200)">return</span> <span style="color: rgb(200,100,0)">false</span>;<br /> <span style="color: rgb(128,128,0)">//</span> <span style="color: rgb(128,128,0)">Check</span> <span style="color: rgb(128,128,0)">to</span> <span style="color: rgb(128,128,0)">see</span> <span style="color: rgb(128,128,0)">if</span> <span style="color: rgb(128,128,0)">a</span> <span style="color: rgb(128,128,0)">number</span> <span style="color: rgb(128,128,0)">of</span> <span style="color: rgb(128,128,0)">functions</span> <span style="color: rgb(128,128,0)">and</span> <span style="color: rgb(128,128,0)">elements</span> <span style="color: rgb(128,128,0)">are<br /></span> <span style="color: rgb(128,128,0)">//</span> <span style="color: rgb(128,128,0)">able</span> <span style="color: rgb(128,128,0)">to</span> <span style="color: rgb(128,128,0)">be</span> <span style="color: rgb(128,128,0)">accessed<br /></span> <span style="color: rgb(0,0,200)">if</span> ( <span style="color: rgb(0,100,200)">document</span> && <span style="color: rgb(0,100,200)">document</span>.getElementsByTagName &&<br /> <span style="color: rgb(0,100,200)">document</span>.getElementById && <span style="color: rgb(0,100,200)">document</span>.body ) {<br /> <span style="color: rgb(128,128,0)">//</span> <span style="color: rgb(128,128,0)">If</span> <span style="color: rgb(128,128,0)">they're</span> <span style="color: rgb(128,128,0)">ready,</span> <span style="color: rgb(128,128,0)">we</span> <span style="color: rgb(128,128,0)">can</span> <span style="color: rgb(128,128,0)">stop</span> <span style="color: rgb(128,128,0)">checking<br /></span> <span style="color: rgb(0,100,200)">clearInterval</span>( domReady.timer );<br /> domReady.timer = <span style="color: rgb(200,100,0)">null</span>;<br /> <span style="color: rgb(128,128,0)">//</span> <span style="color: rgb(128,128,0)">Execute</span> <span style="color: rgb(128,128,0)">all</span> <span style="color: rgb(128,128,0)">the</span> <span style="color: rgb(128,128,0)">functions</span> <span style="color: rgb(128,128,0)">that</span> <span style="color: rgb(128,128,0)">were</span> <span style="color: rgb(128,128,0)">waiting<br /></span> <span style="color: rgb(0,0,200)">for</span> ( <span style="color: rgb(0,0,200)">var</span> i = 0; i < domReady.ready.<span style="color: rgb(0,100,200)">length</span>; i++ )<br /> domReady.ready[i]();<br /> <span style="color: rgb(128,128,0)">//</span> <span style="color: rgb(128,128,0)">Remember</span> <span style="color: rgb(128,128,0)">that</span> <span style="color: rgb(128,128,0)">we're</span> <span style="color: rgb(128,128,0)">now</span> <span style="color: rgb(128,128,0)">done<br /></span> domReady.ready = <span style="color: rgb(200,100,0)">null</span>;<br /> domReady.done = <span style="color: rgb(200,100,0)">true</span>;<br /> }<br />}</pre><br /><p>Notice that in the domReady function, isDOMReady is added as a handler of the "load" event of window. The purpose of this is to take advantages of browser caching cabability to gain extra speed. However, the extra gain here causes troubles. When I tried to use this domReady implementation in a GreaseMonkey user script, sometimes the onDOMReady handler gets triggered twice. It isn't always reproducable. But if you refresh the page 15 times, you can see the double triggering problem one or twice. The only possible cause is the addEvent line. So I commented out the line and tested again, as expected, everything went OK. </p><br /><p>I looked carefully at the code to find a possible race condition in function isDOMReady. The function <br /><ol><br /><li>Checks domReady.done <br /><li>ClearInterval and call handlers if DOM is ready <br /><li>Mark domReady.done true </li></ol>When a page gets cached by browser, the window "load" event and an interval event almost occur at the same time, resulting two threads of isDOMReady executing side by side. In case that one thread is in step 2 but before step 3 while the other is reaching step 1, the later will read domReady.done as false and proceed to step 2, causing every handler triggered a second time. <br /><p>There are two ways to work around <br /><ol><br /><li>Remove the addEvent line and be happy without the extra speed gain <br /><li>Advance the domReady.done = true; line as early as possible (may reduce but can't eliminate race conditions)</li></ol><br /><h4>Update Tue, 06 Feb 2007 09:16:03 GMT window.onload reopened</h4><br /><p>The domReady() function above will prematurelly execute the handler if document.write() is used. The document ready solution in jQuery is so far the most robust. But in IE, premature execution will occur if innerHTML modification is performed before the document finishes loading. So the window.onload problem is now reopened. Great effort has been made to solve the problems.</p><br /><ul><br /><li><a href="http://dean.edwards.name/weblog/2005/09/busted/">The window.onload Problem - Solved!</a> <br /><li><a href="http://dean.edwards.name/weblog/2005/09/busted2/">window.onload - An Alternative Solution </a><br /><li><a href="http://dean.edwards.name/weblog/2006/06/again/">window.onload (again)</a> <br /><li><a href="http://www.outofhanwell.com/blog/index.php?title=the_window_onload_problem_revisited">The window.onload Problem Revisited</a> <br /><li><a href="http://peter.michaux.ca/article/553">The window.onload problem (still)</a> //a very complete coverage</li></ul>Arrixhttp://www.blogger.com/profile/11483857651210475107noreply@blogger.com0tag:blogger.com,1999:blog-2736327227883026435.post-6287735363535493382007-01-19T22:02:00.001+08:002007-06-27T16:30:02.712+08:00JavaScript closure and IE memory leak<p>2007-06-27 08:26:49 UTC Update: Microsoft has fixed IE memory leaks problem. <a href="http://support.microsoft.com/kb/929874/">KB929874</a></p> <p>2006-10-24 +8 Update: According to <a href="http://www.zimbra.com/blog/archives/2006/10/ie_7_a_better_b.html#more">IE 7 vs IE 6</a>, IE 7 seems to have solved the memory leaks. Cheers!</p> <p><strike>2006-09-24 +8 Update: See </strike><a href="http://anotherblog.spaces.live.com/blog/cns!E9C5235EBD2C699D!460.entry"><strike>the follow up</strike></a></p> <p>"Betty: Your umbrella leaks, Professor Boffin!" ---- Look, Listen and Learn</p> <p>IE leaks memory like a sieve and my web page is getting slower and slo...ower. But memory usage keeps climbing...</p> <p>I've tried everything including banging my head on the desk. It just doesn't change anything.</p> <p>I read the following articles and fell asleep.</p> <ul> <li><a href="http://www.bazon.net/mishoo/articles.epl?art_id=824">IE: WHERE'S MY MEMORY?</a> <li><a href="http://blog.morrisjohns.com/javascript_closures_for_dummies">JavaScript Closures for Dummies</a> <li><a href="http://www.jibbering.com/faq/faq_notes/closures.html">JavaScript Closures</a> <li><a href="http://laurens.vd.oever.nl/weblog/items2005/closures/">Leak Free JavaScript Closures</a> <li><a href="http://www.codeproject.com/jscript/leakpatterns.asp">Memory Leakage in Internet Explorer - revisited</a> <li><a href="http://dean.edwards.name/weblog/2005/10/add-event/">addEvent() - My Solution</a> <li><a href="http://msdn.microsoft.com/library/default.asp?url=/library/en-us/IETechCol/dnwebgen/ie_leak_patterns.asp">Understanding and Solving Internet Explorer Leak Patterns</a> <li><a href="http://siteexperts.spaces.live.com/Blog/cns!1pNcL8JwTfkkjv4gg6LkVCpw!338.entry">Closures and IE Circular References</a> <li><a href="http://blogs.msdn.com/ericlippert/archive/2003/09/17/53028.aspx">Fabulous Adventures In Coding</a></li></ul> <p>The fact is that IE has separate garbage collecting mechanisms for COM interfaces and JavaScript objects and is unable to resolve circular references between DOM(or ActiveX or any kind of COM) objects and JavaScript objects. When objects form the two worlds have circular references between them, GC can not detect them and the memory cannot be reclaimed until IE exists. There are several patterns that cause circular references. Unfortunately, assigning nested functions as event handlers falls into this category. IE is rejecting the use of closure, one of the most powerful and flexible feature of JavaScript.</p> <p>It is frustrating to know that experts at Microsoft like Eric Lippert suggest "<em><strong>Don't use closures unless you really need closure semantics. In most cases, non-nested functions are the right way to go.</strong>"</em> It sounds like that programmers are abusing closures, completely ignoring that IE has a big problem with closures. It would be relieving and reasonable to expect that IE will reclaim leaked memory after a page has been unloaded. But that's not the fact. And here I found some explanation: <em><strong>"...the application compatibility lab discovered that there were actually web pages that broke when those semantics were implemented. (No, I don't know the details.) The IE team considers breaking existing web pages that used to work to be way, way worse than leaking a little memory here and there, so they've decided to take the hit and leak the memory in this case."</strong></em> I don't understand. Maintaining backward compatibility with rare web pages which even an scripting engine writer does not know much about at the cost of leaking memory possibly for all web pages? What kind of decision is it? They don't admit their own faults but instead suggest poor coding practices.</p> <p>But we have to code for IE, however buggy it is. I've run the test from Mihai Bazon(see <a href="http://www.bazon.net/mishoo/articles.epl?art_id=824">IE: WHERE'S MY MEMORY?</a>).</p><pre class="code"><span style="color: rgb(0,0,200)">function</span> createEl(i) { <br> <span style="color: rgb(0,0,200)">var</span> span = <span style="color: rgb(0,100,200)">document</span>.createElement(<span style="color: rgb(0,128,0)">"span"</span>); <br> span.className = <span style="color: rgb(0,128,0)">"muci"</span>; <br> span.innerHTML = <span style="color: rgb(0,128,0)">"&nbsp;foobar #"</span>+i+<span style="color: rgb(0,128,0)">"&nbsp;"</span>; <br> span.onclick = <span style="color: rgb(0,0,200)">function</span>() { <br> <span style="color: rgb(0,100,200)">alert</span>(<span style="color: rgb(0,0,200)">this</span>.innerHTML + <span style="color: rgb(0,128,0)">"\n"</span> + i); <br> }; <br> <span style="color: rgb(0,100,200)">document</span>.body.appendChild(span); <br>} <br><br><span style="color: rgb(0,0,200)">function</span> start() { <br> <span style="color: rgb(0,0,200)">var</span> T1 = (<span style="color: rgb(0,0,200)">new</span> <span style="color: rgb(0,100,200)">Date</span>()).getTime(); <span style="color: rgb(128,128,0)">//</span> <span style="color: rgb(128,128,0)">DEBUG.PROFILE</span> <br> <span style="color: rgb(0,0,200)">for</span> (<span style="color: rgb(0,0,200)">var</span> i = 0; i < 3000; ++i) createEl(i); <br> <span style="color: rgb(0,100,200)">alert</span>(((<span style="color: rgb(0,0,200)">new</span> <span style="color: rgb(0,100,200)">Date</span>()).getTime() - T1) <span style="color: rgb(180,0,0)">/ 1000); /</span>/ DEBUG.PROFILE <br>} <br></pre><a href="http://11011.net/software/vspaste"></a><br><br /><p>The first request in IE took 1.797s, the tenth 8.063s. Memory usage kept growing. Firefox reports a typical value of 1.6xs with no memory leak. Prototype.js avoids IE memory leak by hooking window's unload event, unobserving all events and clearing its event handler cache. I replaced the line <em>span.onclick = function() { alert(this.innerHTML + "\n" + i); };</em> with <em>Event.observe(span, 'click', function() { alert(this.innerHTML + "\n" + i); });</em> and rerun the test. The good news is that the leaked memory in IE is reclaimed when the page unloads. The bad news is that each request takes approximately 17s in IE while Firefox only needs 2.1xs! <strong>The Prototype event system makes it possible to free memory when page unloads but is extremely slow and uses more memory in a single request.</strong> The speed degradation is explainable: Event._observeAndCache saves extra references thus uses more memory and IE gets slow as it leaks memory. Event.observe does more things than a simple assignment thus is much slower. However, memory leak is under control... I admire <a href="http://dean.edwards.name/">Edward Dean</a>'s <a href="http://dean.edwards.name/weblog/2005/10/add-event/">addEvent</a> though it doesn't solve the memory leak problem with closures. (Dean insisted that his script does not leak memory in comments. Maybe he is not talking about the closure case). <a href="http://laurens.vd.oever.nl/weblog/items2005/closures/">Leak Free JavaScript Closures</a> solution can really prevent memory leak. <strike>The way it breaks circular reference inserting a new closure between the nested function and its closing scope. The fancy part is that when another level of closure is added, the inner closure can still access variables in its initial closing scope indirectly via scope chain without causing circular references between DOM objects and JavaScript objects.</strike> The way it breaks circular reference is create a new function which holds no references to the closing scope.</p><br><br /><p>In the simplest case:</p><pre class="code"><script type=<span style="color: rgb(0,128,0)">"text/javascript"</span>><br><span style="color: rgb(128,128,0)">//Holds</span> <span style="color: rgb(128,128,0)">references</span> <span style="color: rgb(128,128,0)">to</span> <span style="color: rgb(128,128,0)">functions<br></span>__funcs = []; <br><span style="color: rgb(128,128,0)">//The</span> <span style="color: rgb(128,128,0)">code</span> <span style="color: rgb(128,128,0)">fragment</span> <span style="color: rgb(128,128,0)">is</span> <span style="color: rgb(128,128,0)">for</span> <span style="color: rgb(128,128,0)">demonstrative</span> <span style="color: rgb(128,128,0)">purpose</span> <span style="color: rgb(128,128,0)">only</span> <span style="color: rgb(128,128,0)">and</span> <span style="color: rgb(128,128,0)">lacks</span> <span style="color: rgb(128,128,0)">of</span> <span style="color: rgb(128,128,0)">optimization.<br></span><span style="color: rgb(128,128,0)">//Do</span> <span style="color: rgb(128,128,0)">not</span> <span style="color: rgb(128,128,0)">use</span> <span style="color: rgb(128,128,0)">it</span> <span style="color: rgb(128,128,0)">in</span> <span style="color: rgb(128,128,0)">productive</span> <span style="color: rgb(128,128,0)">environment!<br></span><span style="color: rgb(0,100,200)">Function</span>.<span style="color: rgb(0,0,200)">prototype</span>.closure = <span style="color: rgb(0,0,200)">function</span>() {<br> __funcs.push(<span style="color: rgb(0,0,200)">this</span>);<br> <span style="color: rgb(0,0,200)">return</span> <span style="color: rgb(0,0,200)">function</span> () {<br> <span style="color: rgb(0,0,200)">return</span> __funcs[__funcs.<span style="color: rgb(0,100,200)">length</span> - 1].apply(<span style="color: rgb(200,100,0)">null</span>, <span style="color: rgb(0,100,200)">arguments</span>);<br> };<br>}; <br><span style="color: rgb(0,0,200)">function</span> setup() {<br> <span style="color: rgb(0,0,200)">var</span> span = $(<span style="color: rgb(0,128,0)">'span1'</span>);<br> span.bigProperty = <span style="color: rgb(0,0,200)">new</span> <span style="color: rgb(0,100,200)">Array</span>(1000).join(<span style="color: rgb(0,128,0)">'-_-'</span>);<br> span.onclick = <span style="color: rgb(0,0,200)">function</span>() {<br> <span style="color: rgb(0,100,200)">alert</span>(span.bigProperty.<span style="color: rgb(0,100,200)">length</span>);<br> }.closure();<br>}<br>setup();<br></script> </pre><a href="http://11011.net/software/vspaste"><a href="http://11011.net/software/vspaste"></a><br><br /><p>This will not leak memory in IE. The nested function in setup() forms a closure so it's able access span. span.onclick does not refer to the nested function but a newly created function returned by the closure() method of the nested function. The newly created function invokes the nested function via global array __funcs and have no references to the scope of setup(). So there is no circular reference. You may argue that the newly created function is able to call the nested function, it must have some kind of reference to it while the nested function have reference to span via closure, so there will be a circular reference. However, ECMA 262 treat this as a keyword rather than an identifier, this keyword is resolved dependent on execution context in which it occurs, without reference to the scope chain (see <a href="http://www.jibbering.com/faq/faq_notes/closures.html">Javascript Closures</a> ). A global array is used to hold references to closures without modifying the internal [[scope]] object. This is a hack indeed. IE always needs hacks to amend its holes.</p>Arrixhttp://www.blogger.com/profile/11483857651210475107noreply@blogger.com3tag:blogger.com,1999:blog-2736327227883026435.post-58352618936503749272007-01-19T20:45:00.001+08:002007-01-19T21:32:19.214+08:00Converting the arguments object to an Array<p>The arguments object is an Array-like object providing access to arguments passed to a function. When hacking with functions, we frequently need to manipulate and pass around arguments as an Array. The <a title="The Prototype JavaScript library" href="http://www.prototypejs.org/">Prototype</a> library uses it's $A() function to accomplish the conversion, which is intuitive and beautiful.</p><pre><span style="color: #0000ff">Function</span>.<span style="color: #0000ff">prototype</span>.bind = <span style="color: #0000ff">function</span>() {<br /> <span style="color: #0000ff">var</span> __method = <span style="color: #0000ff">this</span>, args = $A(<span style="color: #0000ff">arguments</span>), object = args.shift();<br /> <span style="color: #0000ff">return</span> <span style="color: #0000ff">function</span>() {<br /> <span style="color: #0000ff">return</span> __method.apply(object, args.concat($A(<span style="color: #0000ff">arguments</span>)));<br /> }<br />}</pre><br /><p>Yet there is a another way.</p><pre><span style="color: #008000">//... inside a function definition</span><br /><span style="color: #0000ff">var</span> args = [].slice.call(<span style="color: #0000ff">arguments</span>);<br /><span style="color: #008000"><br>// or var args = Array.prototype.slice.call(arguments);<br>//...</span></pre><br /><p>The slice(start, end) method of an Array object returns a copy of a specified portion of the array. Here we omit the start and end arguments and call slice() upon the arguments object. An array containing all arguments is returned. Then we can modify the array as we need and pass it to the apply() method of functions.</p><br /><p><strong>Notice</strong> that slice() only does a shallow copy and the return value is just an array any without magical behavior of the arguments object. There is no args.callee property. Moreover, if a parameter is listed at the <em>nth</em> position in the parameter list of the function definition, arguments[n] is a synonym for the local variable corresponding to the <em>nth</em> argument. Any change made to arguments[n] will affect the local variable because it's actually modifying the named property of the call object. However, since args is just a shallow copy of arguments, assigning args[n] will not affect the local variable. To demonstrate this,</p><br /><p><span style="color: rgb(0,0,200)">function</span> f (a) {<br> <span style="color: rgb(0,100,200)">alert</span>(<span style="color: rgb(0,128,0)">'a: '</span> + a); <br> <span style="color: rgb(0,0,200)">var</span> args = [].slice.call(<span style="color: rgb(0,100,200)">arguments</span>); <br> <span style="color: rgb(0,100,200)">arguments</span>[0] = <span style="color: rgb(0,128,0)">'Assigning arguments[0] affects a'</span>; <br> <span style="color: rgb(0,100,200)">alert</span>(<span style="color: rgb(0,128,0)">'a: '</span> + a);<br> args[0] = <span style="color: rgb(0,128,0)">'Assigning args[0] does not change a'</span>; <br> <span style="color: rgb(0,100,200)">alert</span>(<span style="color: rgb(0,128,0)">'a: '</span> + a);<br>}<br>f(<span style="color: rgb(0,128,0)">'JavaScript rocks!'</span>);</p><a href="http://11011.net/software/vspaste"></a><br /><p>The three alerts will display "a: <em>JavaScript rocks"</em>, "a: <em>Assigning arguments[0] affects a"</em> and "a: <em>Assigning arguments[0] affects a"</em> in order.</p><br /><p><strong>Tip</strong>: [].slice.call() can be used for any Array like objects, not only limited to the arguments object.</p>Arrixhttp://www.blogger.com/profile/11483857651210475107noreply@blogger.com5