Add support for AJAX content

[shogun70]

Feature suggestion: Mobify only supports capturing the landing-page. However AJAX and PJAX (pushState + AJAX) loaded content could benefit from the same capability.

My proposed functionality for Mobify has the following:

1. HTMLParser - this takes the xhr.responseText (or any string) and parses it to create a HTMLDocument
2. DOMProcessor - applies processing rules to a document
3. Glue - make the parser and processor work for the captured landing page, etc.

HTMLParser

Looking forward, the HTMLParser will be part of the browsers XMLHttpRequest. In IE8 and below it would be nice to enable html5 elements. Also, in IE9 and below the mere creation of <img src> downloads the resource, so it would be necessary to escape @src before parsing.

DOMProcessor

I suppose this contains a bunch of optional convenience rules, and the capability for externally provided processing.

Glue

• I feel that using document.write is a mistake. Mobify generates a DOM - just insert that into the window document. Scripts will need special handling, but that is potentially a win because it allows you to schedule their loading / execution. At least allow this to be controlled externally.
• A responsive work-around to prevent pre-loading of images is to use something like <a href class="image">. People who use this approach might still prefer mobify if it could do in-place DOM-processing (instead of capture-to-external-document-then-rewrite-page).

[jansepar]

@shogun70 thanks for the detailed suggestions! Here are some of my thoughts:

• HTMLParser: So, you would like the ability to fetch a page and create a Document with no resources loaded? We could certainly create a method for that. If you do want to do this though before it is a feature of the library, feel free to fetch the page and use Mobify.Capture.disable to disable the assets on the page, and create the Document object yourself. Also take note that document.write behaves completely differently in IE9 and below, and thus we do not support those devices.
  • document.write: It would be great to figure out how to get around using document.write - it would be much faster, and it would work on older IEs. But in order to guarantee that your site behaves the same as it did before, attempting to do special script handling could break things. We can hopefully dedicate more time to this in the future. If you come up with an awesome solution feel free to open a pull request! :)
  • I'm not sure what your suggestion is for people using a combination of <a href class="image">. With Capturing you no longer need trickery such as that.

[noahadams]

In HTML5, parser added scripts (which in this context means script tags added using document.write()) are the only scripts with a guaranteed execution order.

Script elements added using DOM methods like Node.appendChild() offer no guarantees about the order in which things will run.

Further, using document.write() to output scripts means that those scripts themselves (say, ad tags) can also call document.write() safely, without destroying the document.

[shogun70]

Hi @noahadams.

1. Actually HTML5 does allow you to insert scripts while maintaining execution order.
   See Dynamic Script Execution Order for
   a meandering analysis and labjs for one implementation.

   It is probably more complex and possibly less robust than what mobify currently does.

2. You are correct about document.write() allowing scripts to call document.write()safely. There are work-arounds for this (I've done it myself) but I've heard they aren't robust.

[shogun70]

Hi @jansepar.

I can see that my issue statement didn't provide a very good context for what I was suggesting. I'll do that now.

I guess the main thing I'm trying to say is I don't like the Mobify API. The code does great stuff, but the API is all about capturing the landing page. This is all upside-down - capturing the landing-page is a small part of what Mobify should be. The Responsive DOM Preprocessing is the central thing.

Capturing is important for now only because you can't fetch that landing-page with XMLHttpRequest. But in a pushState managed session every other page is fetched with XHR and doesn't need capturing - just the pre-processing that Mobify facilitates. Probably 99% of the browsers Mobify supports also have pushState.

It is also worth noting that even the landing-page can be fetched with XHR if it is cached. This could be another option given to the site author.

[johnboxall]

@shogun70 - I think this is a great suggestion - it would be awesome to use Mobify.js' transformations with something like PJAX.

I think the immediate next step is to try combining Mobify.js with something like jquery-pjax and see what glue code is required and if there are any blockers - eg. our use of document.write.