Skip to content

File split#700

Merged
vingtetun merged 25 commits intomozilla:masterfrom
arturadib:filesplit
Oct 27, 2011
Merged

File split#700
vingtetun merged 25 commits intomozilla:masterfrom
arturadib:filesplit

Conversation

@arturadib
Copy link
Contributor

Everything should be working, including the worker stuff.

Major topics:

  • All sources now live under src/
  • Final target is build/pdf.js. Use make pdfjs to generate the file, and make watch to monitor file changes and regenerate target automatically. (Only dependency is make and python, already required by our tests).
  • Worker files have been reorganized, and bundled into worker.js. pdf.js detects its environment (master vs. worker) and initializes appropriately
  • All of our API is exposed through the global object PDF (UPDATE: the global namespace is now PDFJS). Everything else is wrapped under a self-executing function
  • Julian's window[] hack has been modified to use the object Shadings

This was done with @brendandahl - if you guys could take a look at this relatively soon that would be helpful, otherwise it might get gitrotten quickly.

Thanks!

UPDATE:

Changing things a bit after discussion with Julian.

  • Making a single build file is now optional, and only used for production
  • When in development mode, all scripts are to be included manually (yep, all those), so everything should work similar to before this file split. This makes it easier to debug the files.
  • When in production, issue make production. The following (comment) tags found in src/pdf.js and web/viewer.html will instruct the script to do its magic: PDFJSSCRIPT_INCLUDE_ALL, PDFJSSCRIPT_INCLUDE_BUILD, and PDFJSSCRIPT_REMOVE, which will respectively cat-combine all src/*.js files, <script>-include build/pdf.js, and remove the corresponding line.
  • No more make watch, as this is now unnecessary
  • make production depends on sed, but that's OK since contributors don't have to mess with the production script.

Some time in the near future the bot will periodically issue make production and update our gh-pages.

@cgjones @vingtetun @notmasteryet @jviereck @wfwalker

src/pdf.js Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why have a variable here? Everything leaks to the global name space, so I don't see the benefits of this one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything leaks to the global name space

What do you mean by "everything"? Did you see the self-executing function below?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The prototypes like TilingPattern are just about getting added to window object, right? So what's the win of using a window.PDF object and place things on it later again?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what's the win of using a window.PDF object and place things on it later again?

I guess this question was asked before you noticed the global wrapper?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it likely that this name might conflict another variable in a page where pdf.js is included? Would it be better to call it PDFJS instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea. PDFJS also makes it easier to search for API calls across files.

@jviereck
Copy link
Contributor

@arturadib, sorry, I've missed reading the comments :(

  • Final target is build/pdf.js. Use make pdfjs to generate the file, and make watch to monitor file changes and regenerate target automatically. (Only dependency is make and python, already required by our tests).

The build/pdf.js file is great for deployment, but should we really build everything in just one file when developing as well? For debugging purpose, it's way easier (and more speedy!) to have a lot of separate files. That also makes it easier to navigate, as you know you're hacking right now on file x, line y but if everything gets compressed into one big file, you don't know the line numbers anymore.

From my own experience, I spend quite some time in the debugger, so we should make sure that interaction is as easy/speedy as possible.

For Bespin/Skywriter, the build process ripped out all the /src/ links and replaced them with with only one src link. Can we do something similar here?

Makefile Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sed-command does not seem to be doing anything unless my sed-fu is completely off. Could you explain what the intention is for this line? Btw in git bash there is no option -E. There is option -e.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like gnu sed and bsd sed aren't the same. -E is interpret regular expression as extended(modern) regular expression. For our purposes we probably don't need that though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I generally use -E as a default rule as I use modern regular expressions. It's not needed here so I removed it.

@arturadib
Copy link
Contributor Author

For Bespin/Skywriter, the build process ripped out all the /src/ links and replaced them with with only one src link.

By "link" do you mean <script> tags?

So if I understand you correctly, in Bespin by default all development is done by including src/ files separately via <script> tags, and for production the build script replaces these tags with only one <script> tag that points to the final target?

@jviereck
Copy link
Contributor

So if I understand you correctly, in Bespin by default all development is done by including src/ files separately via <script> tags, and for production the build script replaces these tags with only one <script> tag that points to the final target?

Yes. You can take a look at it here:

http://hg.mozilla.org/labs/bespin/file/d498b93b0d28/frontend/index.html#

Basically, the build script replaced the <!-- begin script tags --> and <!-- end script tags --> section by one <script> tag that pointed to the compressed js file generated during the build process.

@jviereck
Copy link
Contributor

@arturadib mentioned on IRC that jquery uses the one file for development as well. As far as I can tell from looking at the test/index.hml, it seems like they use the non-build version for testing as well:

https://github.com/jquery/jquery/blob/master/test/index.html#L11

@arturadib
Copy link
Contributor Author

@jviereck Please check out the updated pull request text. Dev mode is the default now.

Also, can you please download this branch and test it out (including enabling workers)? If you want you can test make production too, but I want to make sure the dev version (that is, no make required) is solid.

Thanks!

@brendandahl
Copy link
Contributor

New devmode seems to be working fine for me. Extension, web, and workers work also.

@jviereck
Copy link
Contributor

@pdfjsbot test

@pdfjsbot
Copy link

Processing command test by user jviereck. Queue size: 0

Live script output is available (after queueing is done) at: http://184.73.87.52:8989/2527108.txt

[bot:processed:2527108]

@pdfjsbot
Copy link

ERROR(s) found

Output:

========== Killing any stray processes

========== Cloning pull request repo
Cloning into ....

========== Running 'make lint'
gjslint --nojsdoc  src/canvas.js src/charsets.js src/cidmaps.js src/colorspace.js src/core.js src/crypto.js src/evaluator.js src/fonts.js src/function.js src/glyphlist.js src/image.js src/metrics.js src/obj.js src/parser.js src/pattern.js src/pdf.js src/stream.js src/util.js src/worker.js src/worker_loader.js  web/compatibility.js web/viewer.js test/driver.js examples/helloworld/hello.js extensions/firefox/bootstrap.js extensions/firefox/components/pdfContentHandler.js
26 files checked, no errors found.

========== Merging upstream into pull request clone
CONFLICT (delete/modify): pdf.js deleted in HEAD and modified in 8341579fd37f7120d7058f4841e6bc146566b0f9. Version 8341579fd37f7120d7058f4841e6bc146566b0f9 of pdf.js left in tree.
Auto-merging src/fonts.js
Automatic merge failed; fix conflicts and then commit the result.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
ERROR: Could not merge upstream into pull request. Please resolve conflicts.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!


_____________________________ stderr:

Bot response time: 0.76 mins

@jviereck
Copy link
Contributor

Cloning the branch locally:

    Julian-Vierecks-MacBook-Pro:pdf_arturadib jviereck$ make test
    cd test; \
        python test.py --reftest \
        --browserManifestFile=resources/browser_manifests/browser_manifest.json \
        --manifestFile=test_manifest.json
    Traceback (most recent call last):
      File "test.py", line 579, in 
        main()
      File "test.py", line 566, in main
        browsers = setUp(options)
      File "test.py", line 321, in setUp
        assert not ANAL or os.path.isfile('../build/pdf.js') and os.path.isdir('../.git')
    AssertionError
    make: *** [browser-test] Error 1

make bundle seems to solve this. Should there be a dependency from the make test on make bundle?

@vingtetun
Copy link
Contributor

When in development mode, all scripts are to be included manually
(yep, all those), so everything should work similar to before this file split.
This makes it easier to debug the files.

Do we have to do that by hand everytime? If yes can we find something smart to avoid that?

Makefile Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BUILD_TARGET?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep!

@vingtetun
Copy link
Contributor

Globally this sounds good to me and I agree that we need to merge that soon (and thanks for not requiring to install anything else for packaging the files :))

@arturadib
Copy link
Contributor Author

@vingtetun

Do we have to do that by hand everytime? If yes can we find something smart to avoid that?

I guess you answered that yourself already :)

src/pdf.js Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be good to name this function? Like (function executeSelfPdfJs() {?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, pdfjsWrapper

@kkujala
Copy link
Contributor

kkujala commented Oct 26, 2011

Looks great! Thanks :)

@arturadib
Copy link
Contributor Author

@pdfjsbot test

@pdfjsbot
Copy link

Processing command test by user arturadib. Queue size: 0

Live script output is available (after queueing is done) at: http://184.73.87.52:8989/2534765.txt

[bot:processed:2534765]

@pdfjsbot
Copy link

All tests passed.

Output:

========== Killing any stray processes

========== Cloning pull request repo
Cloning into ....

========== Running 'make lint'
gjslint --nojsdoc  src/canvas.js src/charsets.js src/cidmaps.js src/colorspace.js src/core.js src/crypto.js src/evaluator.js src/fonts.js src/function.js src/glyphlist.js src/image.js src/metrics.js src/obj.js src/parser.js src/pattern.js src/pdf.js src/stream.js src/util.js src/worker.js src/worker_loader.js  web/compatibility.js web/viewer.js test/driver.js examples/helloworld/hello.js extensions/firefox/bootstrap.js extensions/firefox/components/pdfContentHandler.js
26 files checked, no errors found.

========== Merging upstream into pull request clone

========== Cloning reference images repo into test/ref/
Initialized empty Git repository in /home/ubuntu/pdf.js-bot/tmp/tests/41caf5018d455969f437b1dbcb86ef34b371234f/test/ref/.git/

========== Checking for consistency with reference repo

========== Running 'make bot_test'
cd test; \
    python -u test.py \
    --browserManifestFile=resources/browser_manifests/browser_manifest.json \
    --manifestFile=test_manifest.json
Launching firefox
Xlib:  extension "RANDR" missing on display ":1".
TEST-PASS | eq test tracemonkey-eq | in firefox
TEST-PASS | forward-back-forward test tracemonkey-fbf | in firefox
TEST-PASS | load test html5-canvas-cheat-sheet-load | in firefox
TEST-PASS | load test intelisa-load | in firefox
TEST-PASS | load test pdfspec-load | in firefox
TEST-PASS | load test shavian-load | in firefox
TEST-PASS | eq test sizes | in firefox
TEST-PASS | eq test openweb-cover | in firefox
TEST-PASS | eq test plusminus | in firefox
TEST-PASS | load test openoffice-pdf | in firefox
TEST-PASS | load test openofficecidtruetype-pdf | in firefox
TEST-PASS | load test openofficearabiccidtruetype-pdf | in firefox
TEST-PASS | load test arabiccidtruetype-pdf | in firefox
TEST-PASS | load test complexttffont-pdf | in firefox
TEST-PASS | eq test thuluthfont-pdf | in firefox
TEST-PASS | eq test wnv_chinese-pdf | in firefox
TEST-PASS | eq test i9-pdf | in firefox
TEST-PASS | load test hmm-pdf | in firefox
TEST-PASS | eq test rotation | in firefox
TEST-PASS | load test ecma262-pdf | in firefox
TEST-PASS | load test jai-pdf | in firefox
TEST-PASS | eq test cable | in firefox
TEST-PASS | eq test pdkids | in firefox
TEST-PASS | eq test artofwar | in firefox
TEST-PASS | eq test wdsg_fitc | in firefox
TEST-PASS | eq test unix01 | in firefox
TEST-PASS | eq test fit11-talk | in firefox
TEST-PASS | eq test fips197 | in firefox
TEST-PASS | load test txt2pdf | in firefox
TEST-PASS | load test f1040 | in firefox
TEST-PASS | load test hudsonsurvey | in firefox
TEST-PASS | eq test extgstate | in firefox
TEST-PASS | eq test usmanm-bad | in firefox
TEST-PASS | load test vesta-bad | in firefox
TEST-PASS | load test ibwa-bad | in firefox
TEST-PASS | eq test tcpdf_033 | in firefox
TEST-PASS | eq test pal-o47 | in firefox
TEST-PASS | eq test simpletype3font | in firefox

All tests passed.
Process firefox is still running. Killing.
Runtime was 1700 seconds
Exception in thread Thread-1 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
  File "/usr/lib/python2.7/threading.py", line 505, in run
  File "/usr/lib/python2.7/SocketServer.py", line 225, in serve_forever
<type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'select'

========== Cleaning up

All done.


_____________________________ stderr:

Bot response time: 29.20 mins

@vingtetun
Copy link
Contributor

I mostly agree with all this split, so let's do it right now to not block to much on that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants