Webmaster Forum / HTML, CSS, Scripts / JavaScript / August 2007

Peter Michaux said the following on 8/21/2007 10:19 PM:

What you are describing in this thread is more minimization than
obfuscation.

<snip>

That is what most do now, simply remove whitespace. While it isn't true
minimization, it is far from obfuscation.

A false sense of security because it is "obfuscated".

A good example of minimizing versus obfuscating is bookmarklets that do
anything with variables. They get minimized because of character
restraints.

This is a bookmarklet that I wrote recently (You may have read the thread):

javascript:
a=location.href;
b=document.domain;
j="www.live1.com";
k="alt.live1.com";
l="www.live2.com";
m="alt.live2.com";
n="www.live3.com";
p="alt.live3.com";
q="www.live4.com";
r="alt.live4.com";
switch(b){
case j:s=k;break;
case k:s=j;break;
case l:s=m;break;
case m:s=l;break;
case n:s=p;break;
case p:s=n;break;
case q:s=r;break;
case r:s=q;break;
};
a=a.replace(b,s);
location.href=a;

It is minimized to a strong degree because it is for a bookmarklet.
Removing the line breaks it looks like this (all on one line):

javascript:a=location.href;b=document.domain;j="www.live1.com";k="alt.live1.com";l="www.live2.com";m="alt.live2.com";n="www.live3.com";p="alt.live3.com";q="www.live4.com";r="alt.live4.com";switch(b){case
j:s=k;break;case k:s=j;break;case l:s=m;break;case m:s=l;break;case
n:s=p;break;case p:s=n;break;case q:s=r;break;case
r:s=q;break;};a=a.replace(b,s);location.href=a;

That is minimized as much as I care to minimize it.

This is the same script "obfuscated" (from a web search for obfuscators):

eval(unescape('%68%35%3D%30%3B%68%30%3D%27%%36%44%%36%35%%36%45%%37%34%%32%45%%35%35%%35%32%%34%43%%33%42%27%3B%66%75%6E%63%74%69%6F%6E%20%68%31%35%28%68%31%31%29%7B%68%38%3D%4D%61%74%68%2E%72%6F%75%6E%64%28%68%31%31%2F%34%30%39%36%2D%2E%35%29%3B%68%31%33%3D%68%31%31%2D%68%38%2A%34%30%39%36%3B%68%39%3D%4D%61%74%68%2E%72%6F%75%6E%64%28%68%31%33%2F%32%35%36%2D%2E%35%29%3B%68%31%34%3D%68%31%33%2D%68%39%2A%32%35%36%3B%68%31%30%3D%4D%61%74%68%2E%72%6F%75%6E%64%28%68%31%34%2F%31%36%2D%2E%35%29%3B%68%31%36%3D%68%31%34%2D%68%31%30%2A%31%36%3B%72%65%74%75%72%6E%28%27%27%2B%68%31%39%28%68%31%30%29%2B%68%31%39%28%68%31%36%29%29%3B%7D%68%32%3D%27%%34%41%%34%31%%35%36%%34%31%%35%33%%34%33%%35%32%%34%39%%35%30%%35%34%%31%41%%30%30%%32%44%%32%41%%34%31%%31%44%%34%43%%34%46%%34%33%%34%31%%35%34%%34%39%%34%46%%34%45%%30%45%%34%38%%35%32%%34%35%%34%36%%31%42%%30%30%%32%44%%32%41%%34%32%%31%44%%34%34%%34%46%%34%33%%35%35%%34%44%%34%35%%34%45%%35%34%%30%45%%34%34%%34%46%%34%44%%34%31%%34%39%%34
%45%%31%42%%30%30%%32%44%%32%41%%34%41%%31%44%%30%32%%35%37%%35%37%%35%37%%30%45%%34%43%%34%39%%35%36%%34%35%%31%31%%30%45%%34%33%%34%46%%34%44%%30%32%%31%42%%30%30%%32%44%%32%41%%34%42%%31%44%%30%32%%34%31%%34%43%%35%34%%30%45%%34%43%%34%39%%35%36%%34%35%%31%31%%30%45%%34%33%%34%46%%34%44%%30%32%%31%42%%30%30%%32%44%%32%41%%34%43%%31%44%%30%32%%35%37%%35%37%%35%37%%30%45%%34%43%%34%39%%35%36%%34%35%%31%32%%30%45%%34%33%%34%46%%34%44%%30%32%%31%42%%30%30%%32%44%%32%41%%34%44%%31%44%%30%32%%34%31%%34%43%%35%34%%30%45%%34%43%%34%39%%35%36%%34%35%%31%32%%30%45%%34%33%%34%46%%34%44%%30%32%%31%42%%30%30%%32%44%%32%41%%34%45%%31%44%%30%32%%35%37%%35%37%%35%37%%30%45%%34%43%%34%39%%35%36%%34%35%%31%33%%30%45%%34%33%%34%46%%34%44%%30%32%%31%42%%30%30%%32%44%%32%41%%35%30%%31%44%%30%32%%34%31%%34%43%%35%34%%30%45%%34%43%%34%39%%35%36%%34%35%%31%33%%30%45%%34%33%%34%46%%34%44%%30%32%%31%42%%30%30%%32%44%%32%41%%35%31%%31%44%%30%32%%35%37%%35%37%%35%37%%30%45%%34%43%%34%39%%35%36%%34%35
%%31%34%%30%45%%34%33%%34%46%%34%44%%30%32%%31%42%%30%30%%32%44%%32%41%%35%32%%31%44%%30%32%%34%31%%34%43%%35%34%%30%45%%34%43%%34%39%%35%36%%34%35%%31%34%%30%45%%34%33%%34%46%%34%44%%30%32%%31%42%%30%30%%32%44%%32%41%%35%33%%35%37%%34%39%%35%34%%34%33%%34%38%%30%38%%34%42%%30%39%%35%42%%30%30%%32%44%%32%41%%34%33%%34%31%%35%33%%34%35%%30%30%%34%41%%31%41%%35%33%%31%44%%34%42%%31%42%%34%32%%35%32%%34%35%%34%31%%34%42%%31%42%%30%30%%32%44%%32%41%%34%33%%34%31%%35%33%%34%35%%30%30%%34%42%%31%41%%35%33%%31%44%%34%41%%31%42%%34%32%%35%32%%34%35%%34%31%%34%42%%31%42%%30%30%%32%44%%32%41%%34%33%%34%31%%35%33%%34%35%%30%30%%34%43%%31%41%%35%33%%31%44%%34%44%%31%42%%34%32%%35%32%%34%35%%34%31%%34%42%%31%42%%30%30%%32%44%%32%41%%34%33%%34%31%%35%33%%34%35%%30%30%%34%44%%31%41%%35%33%%31%44%%34%43%%31%42%%34%32%%35%32%%34%35%%34%31%%34%42%%31%42%%30%30%%32%44%%32%41%%34%33%%34%31%%35%33%%34%35%%30%30%%34%45%%31%41%%35%33%%31%44%%35%30%%31%42%%34%32%%35%32%%34%35%%34%31%%34%42%%31%42%%3
0%30%%32%44%%32%41%%34%33%%34%31%%35%33%%34%35%%30%30%%35%30%%31%41%%35%33%%31%44%%34%45%%31%42%%34%32%%35%32%%34%35%%34%31%%34%42%%31%42%%30%30%%32%44%%32%41%%34%33%%34%31%%35%33%%34%35%%30%30%%35%31%%31%41%%35%33%%31%44%%35%32%%31%42%%34%32%%35%32%%34%35%%34%31%%34%42%%31%42%%30%30%%32%44%%32%41%%34%33%%34%31%%35%33%%34%35%%30%30%%35%32%%31%41%%35%33%%31%44%%35%31%%31%42%%34%32%%35%32%%34%35%%34%31%%34%42%%31%42%%30%30%%32%44%%32%41%%32%44%%32%41%%35%44%%31%42%%30%30%%32%44%%32%41%%32%44%%32%41%%32%44%%32%41%%34%31%%31%44%%34%31%%30%45%%35%32%%34%35%%35%30%%34%43%%34%31%%34%33%%34%35%%30%38%%34%32%%30%43%%35%33%%30%39%%31%42%%30%30%%32%44%%32%41%%34%43%%34%46%%34%33%%34%31%%35%34%%34%39%%34%46%%34%45%%30%45%%34%38%%35%32%%34%35%%34%36%%31%44%%34%31%%31%42%%30%30%27%3B%66%75%6E%63%74%69%6F%6E%20%68%31%39%28%68%32%31%29%7B%69%66%28%68%32%31%3C%31%30%29%72%65%74%75%72%6E%20%68%32%31%3B%65%6C%73%65%20%72%65%74%75%72%6E%20%53%74%72%69%6E%67%2E%66%72%6F%6D%43%68%61%72%43%6F%64%65
%28%68%32%31%2B%35%35%29%3B%7D%68%34%3D%27%27%3B%68%37%3D%27%%36%38%%33%36%%33%44%%36%34%%36%46%%36%33%%37%35%27%3B%68%31%3D%68%32%2E%73%70%6C%69%74%28%27%%27%29%3B%65%76%61%6C%28%75%6E%65%73%63%61%70%65%28%68%37%2B%68%30%29%29%3B%68%31%37%3D%27%%36%35%%37%36%%36%31%%36%43%%32%38%%37%35%%36%45%%36%35%%37%33%%36%33%%36%31%%37%30%%36%35%%32%38%%36%38%%33%34%%32%39%%32%39%27%3B%66%6F%72%28%68%33%3D%31%3B%68%33%3C%68%31%2E%6C%65%6E%67%74%68%3B%68%33%2B%2B%29%7B%68%34%2B%3D%75%6E%65%73%63%61%70%65%28%27%%27%2B%68%31%35%28%75%6E%65%73%63%61%70%65%28%27%%27%2B%68%31%5B%68%33%5D%29%2E%63%68%61%72%43%6F%64%65%41%74%28%30%29%5E%68%36%2E%63%68%61%72%43%6F%64%65%41%74%28%68%35%29%29%29%3B%68%35%2B%2B%3B%69%66%28%68%35%3D%3D%68%36%2E%6C%65%6E%67%74%68%29%68%35%3D%30%3B%7D%3B%65%76%61%6C%28%75%6E%65%73%63%61%70%65%28%68%31%37%29%29'));

It includes a test for the path to the file but the effect is there.

I always worry about the notion of (self or externally) imposed blanket
restrictions on what may be done in javascript. For a start we see too
many questions on this group that take the form "I have tied my own
hands and now I need to know how to do the things that became imposable
when I tied my own hands", which don't tend to get what the OP might
consider 'good' answers.

There certainly are some things that everyone should be avoiding, and
the recent illustration of the non-obvious/counterintuitive consequences
of the use of the - with - statement in javascript goes a long way
towards justifying a general assertion that the - with - statement
"should not be used in javascript". But then there is a justifiable use
of the - with - statement, to deliberately augment the scope chain of a
dynamically created function object. The need to do that is rare
(pushing exceptional) but it does exist. Would you want to sacrifice
that possibility just because the rest of the time the - with -
statement "should not be used in javascript"?

The - with - statement, along with - eval - (which is itself subject to
similar injunctions on its use) are the two constructs that can really
mess-up poor obfuscation/minimisation methods.

An illustration of - eval -'s role came to my attention earlier in the
week. For some reason (curiosity as to how they had achieved their
latest spectacular f**k-up) I took a look at one of the javascript files
imported by Google groups pages. The file is -
g2_common-1cb0e454e081b7db50b59661d9bad546.js - (though I suspect that
all the characters after the dash change regularly (they look like a
session ID)), and that file has been 'minimised' using a technique that
has attempted to transform local Identifier names to shorter (one
character) names (a good idea for a minimiser, though a sub-optimal
implementation here). In that code I found (and it is quite easy to find
as it appears twice) a - Function.prototype.appply - emulation (a
subject I have written about many times in the past) which is presumably
to all support on pre-JScritp 5.6 IE browsers (which means mostly IE 5
to 5.5 as that is ActiveX became scriptable). The code reads
(re-wrapped):-

It is reasonably obvious what that code is trying to do (so much for
obscurity) and it is pretty obvious that the process of minimisation had
broken it, by transforming the Identifiers - oScope - and - args - into
one character equivalents but not doing so inside the strings that will
be - eval -ed. And of course a minimiser should not be messing with the
contents of string literals, but what it should be doing is recognising
the - eval - use and not trying to transform any Identifiers in the
(lexically) containing scopes.

Though a better solution would be observing that you only need to use -
eval - in a truly general - apply - emulation, while no real script
contexts are truly general so there was never a need to use - eval -
here in the first place. Still, if Googl's script authors are not with
it enough to notice that they have included the - appply - emulation
twice in the same script I suppose it is a bit unrealistic to expect
them underside what they are doing or why they are doing it.

If conditional comments were a rare, unknown or unexpected feature of
web authoring then you would be right. But we know, if only form
observation, that that is not the case. It may be a pain to take
something like Rhino, for which a comment is just a comment, and make it
see when a comment is a conditional comment (especially when Microsoft
is not going to make the conditional comment algorithms public), but
that is a job that has to be done before the end result is going to be
of general use.

You switch between obfuscation and minimisation at the drop of a hat.

Yes, but where has this one drifted to? are we talking about code
minimisation or code obfuscation at this point?

A degree of obfuscation may be a likely side-effect of minimisation, but
it is not the goal of minimisation. While minimisation is only a
possible side effect of obfuscation. There is, for example, a heavily
marketed commercial script obfuscator that attempts to increase
obscurity by transforming Identifiers into sequences of characters that
look numeric (upper case - o - and lower case - L - followed by decimal
digits), the result increases obscurity (at least in the absence of
syntax highlighting) by often does nothing to reduce code size (on its
own).

That is not true of many obfuscators I have looked at, and attributing
the motivation of code size reduction to the transformations performed
by an obfuscator would be questionable. If software is designed to
obscure the likely primary motivation for any actions it takes are
likely to be increased obscurity not minimisation.

A level of the one can be the side effect of the other, in both
directions.

<snip>

So is "minification" (which implies an attempt to achieve an extreme of
small-ness) the only thing that will prevent commented source code from
being distributed with its comments? Obviously not.

Well, you would think so, but reality is disappointing in that area.

<snip>

<snip>

Where would you attribute the fault in the Google code I posted above?
It is being distributed to client browsers, and it is broken. It may
have been broken by the attempt at minimisation, but it was a failure to
run proper QA tests on the post-minimisation version that has resulted
in its being distributed in its broken state.

Yes, that is fairly necessary if minimisation is to go beyond removing
non-significant white space and comments. On the other hand, removing
non-significant white space and comments will invariably have a big
impact on code size (and a side-effect on relative obscurity). While
interpreting faces the problem that some javascript is (or may be)
interpreted at runtime.

<snip>

Yes, at least when they know enough to make the judgment.

The "general public"? What do the general public have to do with browser
scripting?

But it would not be reasonable if you could not send ECMAScript to a
browser supporting either (that is, after all, the point of having a
standardised language).

Only most?

Any library claiming to be "good for everything" would be a bad library
for that reason alone, as there a no real world contexts where
everything is needed. (Considering that the project I am working on at
the moment now has a client-side code base of over 100,000 lines and it
certainly does not need to do 'everything', for example, there is
currently no code at all that attempts to manipulate user selections)

Does it still, or are the mistakes of the past being corrected?

Seeing what was being done with it I was not at all surprised that the
people making Rails design decisions were also 'approving' poor script
code.

No, some minimisers work only at the source text level, they may take in
what is effetely a program, and then output what is still effetely a
program, but they are not compilers.

Preferably informed judgments.

And Google groups think they are supporting JScript < 5.6 but in reality
they are not. and they don't know they are not because they did not do
the QA.

<snip>

The problem with obfuscators is exactly the same problem it has always
been; they do not achieve a sufficient level of
(non-trivially-reversible) obscurity to provide anything resembling
'protection' ("moderate protection" or otherwise). The formatting and
non-significant white space can be resorted with the simplest of tools,
the DOM and browser object model interactions cannot be subject to
Identifier modification because that will break them, the names of
javascript object properties cannot be modified unless you have
simultaneous access to all the code in the entire system when
obfuscating because you could not guarantee that other code will not
want to refer to those property names, the global identifiers cannot be
modified for the same reasons (under the same circumstances), and that
just leaves the local parameters, inner function declaration and
variable names amenable to modification (and only if they are in scopes
where - with - and - eval - are never used).

And what happens when you change those local Identifiers? There
meaningful names are gone, but their names where maybe not meaningful to
someone who did not understand the author's native language to start
with, and as each Identifier was only meaningful within a confined
lexical scope anyway it does not take much looking to work out their
roles in thier context without ever knowing what (possibly) meaningful
names they may have had. Indeed, if a local variable needed a name that
was any more than unique in its scope then no non-English
speaking/reading programmer could never learn anything from most of the
code examples that exist. We would live in a world where software
authoring was virtually restricted to the English speaking nations, and
that is just not the case.

Minimisation and obfuscation are not the same thing, where the goal is
obfuscation the outcome will not justify the effort. Minimisation may
have a different cost-benefit, but Google are doing a good job of
illustration that the cost side of the equation may not always be being
properly perceived.

Richard.