Comments on: Node.js, YUI 3 & Dom Manipulation… Oh My!

By: Dav Glass

Dav Glass — Mon, 07 Mar 2011 13:45:16 +0000

@Pete –

You should use Node 0.4.2 and Express@1.0.7 if you want to use my examples. The Express beta is too new and I haven’t moved my examples over to it yet.

By: pete

pete — Mon, 07 Mar 2011 00:07:04 +0000

this is so friggin cool and i can’t wait to see it in operation but…

i can’t get the project to run. the debugger tells me that server.js bombs on line 35 as express.bodyDecoder() is not defined.

i’m a heavy user of YUI and less of RoR and wanted to see how node.js and serverside YUI could help me. i never used node.js before so don’t know if the version i have is the problem (node.js 0.5.0-pre and express 2.0.0beta).

pete

By: Dav Glass

Dav Glass — Mon, 20 Dec 2010 20:11:38 +0000

@Sai –

Can you email them to me instead of posting here, it’s hard to read in the comments ;)

davglass (at) gmail (dot) com

By: Sai

Sai — Mon, 20 Dec 2010 09:58:55 +0000

I went back and installed Nave to switch between node versions, but ran into the same errors as earlier. these are the examples that do not work for me:

scrape.js, scrape-object.js, tnt-datatable-script.js, y-server.js.

The errors are consistently related to setters on null or setters on objects that do not have a setter.

By: Sai

Sai — Mon, 20 Dec 2010 08:37:20 +0000

I’m sorry for the dumb observation above….all the examples have the YUI object as a callback param in use.

Here is the trace

./tnt-datatable-script.js
info: (yui): Modules missing: nodejs-dom, 1
info: (yui): Fetching loader: /usr/local/lib/node/.npm/yui3-core/2010.11.03/package/build/loader/loader-debug.js
info: (get): URL: /usr/local/lib/node/.npm/yui3-core/2010.11.03/package/build/loader/loader-debug.js
(node) process.compile should not be used. Use require(‘vm’).runInThisContext instead.
info: (get): Loaded: /usr/local/lib/node/.npm/yui3-core/2010.11.03/package/build/loader/loader-debug.js
info: (yui): Modules missing: dump,nodejs-dom, 2
info: (yui): Using Loader
info: (loader): attempting to load dump, /usr/local/lib/node/.npm/yui3-core/2010.11.03/package/build/
info: (get): URL: /usr/local/lib/node/.npm/yui3-core/2010.11.03/package/build/dump/dump-debug.js
info: (get): Loaded: /usr/local/lib/node/.npm/yui3-core/2010.11.03/package/build/dump/dump-debug.js
info: (loader): attempting to load nodejs-dom, /usr/local/lib/node/.npm/yui3-core/2010.11.03/package/build/
info: (get): URL: /usr/local/lib/node/.npm/yui3/0.5.17/package/lib/yui3-dom.js
info: (get): Loaded: /usr/local/lib/node/.npm/yui3/0.5.17/package/lib/yui3-dom.js
info: (loader): loader finishing: success, yui_3_2_0_1_12928339018771, yui-base,yui-log,dump,nodejs-dom
info: (yui): Attaching available dependencies: yui-base,yui-log,dump,nodejs-dom
info: (yui): Nested use callback: yui-base,yui-log,dump,nodejs-dom
info: (yui): caching request: nodejs-dom
info: JSDom testing..
info: (loader): Undefined module: yui2-datatable, matched a pattern: yui2-
info: (loader): Undefined module: yui2-datasource, matched a pattern: yui2-
info: (yui): Modules missing: yui2-datatable,yui2-datasource, 2
info: (yui): Using Loader
info: (loader): attempting to load yui2-datatable, /usr/local/lib/node/.npm/yui3-core/2010.11.03/package/build/
info: (get): URL: http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-datatable/yui2-datatable-debug.js
info: (get): Loaded: http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-datatable/yui2-datatable-debug.js
info: (loader): attempting to load yui2-datasource, /usr/local/lib/node/.npm/yui3-core/2010.11.03/package/build/
info: (get): URL: http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-datasource/yui2-datasource-debug.js
info: (get): Loaded: http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-datasource/yui2-datasource-debug.js
info: (loader): loader finishing: success, yui_3_2_0_1_12928339018771, yui2-datatable,yui2-datasource
info: (loader): Undefined module: yui2-yahoo, matched a pattern: yui2-
info: (loader): Undefined module: yui2-dom, matched a pattern: yui2-
info: (loader): Undefined module: yui2-event, matched a pattern: yui2-
info: (loader): Undefined module: yui2-skin-sam-datatable, matched a pattern: yui2-
info: (loader): Undefined module: yui2-element, matched a pattern: yui2-
info: (yui): Modules missing: yui2-yahoo,yui2-event,yui2-dom,yui2-skin-sam-datatable,yui2-element, 5
info: (yui): Using Loader
info: (loader): attempting to load yui2-skin-sam-datatable, /usr/local/lib/node/.npm/yui3-core/2010.11.03/package/build/
debug: (get): Get.css
debug: (get): Real CSS loading
info: (loader): attempting to load yui2-yahoo, /usr/local/lib/node/.npm/yui3-core/2010.11.03/package/build/
info: (get): URL: http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-yahoo/yui2-yahoo-debug.js
info: (get): Loaded: http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-yahoo/yui2-yahoo-debug.js
info: (loader): attempting to load yui2-event, /usr/local/lib/node/.npm/yui3-core/2010.11.03/package/build/
info: (get): URL: http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-event/yui2-event-debug.js
info: (get): Loaded: http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-event/yui2-event-debug.js
info: (loader): attempting to load yui2-dom, /usr/local/lib/node/.npm/yui3-core/2010.11.03/package/build/
info: (get): URL: http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-dom/yui2-dom-debug.js
info: (get): Loaded: http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-dom/yui2-dom-debug.js
info: (loader): attempting to load yui2-element, /usr/local/lib/node/.npm/yui3-core/2010.11.03/package/build/
info: (get): URL: http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-element/yui2-element-debug.js
info: (get): Loaded: http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-element/yui2-element-debug.js
info: (loader): loader finishing: success, yui_3_2_0_1_12928339018771, yui2-yahoo,yui2-event,yui2-datasource,yui2-dom,yui2-skin-sam-datatable,yui2-element,yui2-datatable
info: (yui): Nested use callback: yui2-datatable,yui2-datasource
info: (yui): caching request: yui2-datatable,yui2-datasource
info: Creating DataSource..
info: Creating DataTable..
info: (get): Loaded: http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-element/yui2-element-debug.js
error: (nodejsyui3): ———————————————————-
error: (nodejsyui3): TypeError: Cannot set property headers of # which has only a getter
error: (nodejsyui3): at [object Object]._formatTdEl (http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-datatable/yui2-datatable-debug.js:5480:18)
error: (nodejsyui3): at [object Object]._getTrTemplateEl (http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-datatable/yui2-datatable-debug.js:5448:25)
error: (nodejsyui3): at [object Object]._addTrEl (http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-datatable/yui2-datatable-debug.js:5519:29)
error: (nodejsyui3): at [object Object]. (http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-datatable/yui2-datatable-debug.js:7327:47)
error: (nodejsyui3): at Object.run (http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-datatable/yui2-datatable-debug.js:92:28)
error: (nodejsyui3): at [object Object]._runRenderChain (http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-datatable/yui2-datatable-debug.js:7269:24)
error: (nodejsyui3): at [object Object].render (http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-datatable/yui2-datatable-debug.js:7397:10)
error: (nodejsyui3): at [object Object]. (http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-datatable/yui2-datatable-debug.js:13101:18)
error: (nodejsyui3): at Function.issueCallback (http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-datasource/yui2-datasource-debug.js:389:26)
error: (nodejsyui3): at [object Object].handleResponse (http://yui.yahooapis.com/2in3.4/2.8.2/build/yui2-datasource/yui2-datasource-debug.js:1150:8)
error: (nodejsyui3): ———————————————————-

By: Sai

Sai — Mon, 20 Dec 2010 08:25:34 +0000

Dav

I’m running your express server example on node v 0.3.2 (the example that combines everything with a left nav). It fails on the first set title call as “cannot call set method of null”.

I can run the individual calendar and tabview examples, but not the datatable example or the scrape example.

I guess anywhere I get the YUI object as a callback param is where its failing right now

Any ideas/gotchas? Is this node 0.3 issue?

By: Jorge Ventura

Jorge Ventura — Sat, 04 Dec 2010 19:36:02 +0000

Dav,
I was checking your code and I think that I can complete some parts. My purpose is to scrape always the same application, I don’t care about images, CSS and the user interface (buttons, click e etc), I need only to generate the DOM structure correctly.

About the scripts, I can run the code, after load to complete the DOM inside the page. I have to implement the methods that make changes in the DOM as document.write and document.writeln.

Maybe necessary to implement window.open and the Browser must have a window list because the scripts at run time will demand for new windows.

Do you have any suggestion to make my life easier?

Thank you,
Ventura

By: Dav Glass

Dav Glass — Sat, 04 Dec 2010 03:58:26 +0000

@Jorge –

You can access the content of a remote page like this:
https://gist.github.com/574000

However, you can not guarantee that the remote page will execute Javascript and you will not have access to the page to “click links” or interact with it. It will merely be a local modifiable copy of the requested page.

By: Jorge Ventura

Jorge Ventura — Sat, 04 Dec 2010 01:19:37 +0000

Ooops!
Should be

window.location=‘http://www.ajax_application.any’;

By: Jorge Ventura

Jorge Ventura — Fri, 03 Dec 2010 03:32:03 +0000

I read this article and have a question: May I use this stuffs as a headless browser ?

For example the code bellow will works ?

YUI().use(‘nodejs-dom’, ‘event’, ‘node’, function(Y) {
var document = Y.Browser.document;
document.location = ‘http://www.ajax_application.any’;
sys.puts(document.outerHTML);
});

I have a complex application that I would like to scrape using something like this. Right now I am using Envjs but I know that I will have performance penalties in production and I would like to get a better option.

Ventura