Ohh, nice! Works great with Python 2.7.

Note, it doesn't work in Python 3:

----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 52224)
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/socketserver.py", line 306, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/socketserver.py", line 332, in process_request
    self.finish_request(request, client_address)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/socketserver.py", line 345, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/socketserver.py", line 666, in __init__
    self.handle()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/http/server.py", line 400, in handle
    self.handle_one_request()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/http/server.py", line 388, in handle_one_request
    method()
  File "/Users/tom/Library/Python/3.3/lib/python/site-packages/astropy-0.3.dev5047-py3.3-macosx-10.8-x86_64.egg/astropy/table/table.py", line 1313, in do_GET
    self.wfile.write(html[i:i+bufferSize])
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/socket.py", line 317, in write
    return self._sock.send(b)
TypeError: 'str' does not support the buffer interface
----------------------------------------

Just a suggestion - maybe the web browser version could include some CSS to add light lines between cells (excel-style)?

Yes, CSS formatting would be nice, I'll add that as an option. Maybe an
additional header too...

Any thoughts on how to test this? Will Travis behave sanely with
webbrowser?
On Aug 13, 2013 10:53 AM, "Thomas Robitaille" notifications@github.com
wrote:

Just a suggestion - maybe the web browser version could include some CSS
to add light lines between cells (excel-style)?

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/1342#issuecomment-22579754
.

Also, I don't think I understand the python3 error - is it complaining that I'm trying to write unencoded strings?

I don't think any more formatting work should be added on this PR without going a whole next level of making a "table browser" class. I could imagine creating a class where you build in your own css formatting, header, etc, and maybe even building in javascript callbacks for, e.g., sorting.

The CSS doesn't seem to work here (will examine why later). Also, my feeling is that the server, once created, should live until the Table is garbage collected, otherwise doing e.g. 'refresh' causes the page to disappear.

+1 to the idea of making an HTML-based table browser!

@taldcroft might want to see this.

Agreed, the webbrowser should live for a while... when should it be killed, though? I'm not sure there's a good way to make that decision, except perhaps to let it die when the python session dies. I think the latest commit will make that happen for most use-cases though.

Here's an example:

from astropy import units as u
from astroquery.splatalogue import Splatalogue
CO = Splatalogue.query_lines(115.271*u.GHz,800*u.GHz,chemical_name=' CO ')
CO.show_in_browser()

This is what it looks like locally:

I think the only way to have the server live forever is to have it run in its own thread, correct? Dealing with threads, particularly in a way that works will in a library usable in a number of contexts is going to be very hard to get right. I'd say let's avoid that and keep the scope of this to something that sends the data to the browser once and then immediately goes away. Also, keeping it around for the whole session would also imply keeping a reference to the data for the whole session, possibly leading to a "memory leak" (in quotes because it's just an unintentional keeping of references, not an uncollectable memory leak in the traditional sense).

Ok, then in that case would it not make more sense to write to a temporary file and have the browser read that, so that actions such as refreshing don't automatically cause the page to disappear?

Yes, I think that may make more sense. However, when I tried using NamedTemporaryFile, it did the cleanup upon function return. If I instead do NamedTemporaryFile(delete=False), we have the same "memory leak" issue.

I would also prefer a temp file over trying to launch a web server, which is fraught with more issues than is immediately apparent.

I'd also prefer to see a valid HTML5 document--add a <!DOCTYPE html>, and also consider putting the style in a <head> section and the table in a <body> section at a minimum. I would also consider adding a _repr_html_ for use in IPython Notebook. I had assumed we already had that but I don't see it now.

@embray how's that?
(edit: I mean, how do these last two commits look?)

FYI, I only used the WebServer because I couldn't get NamedTemporaryFile working; returning it solved the issue I had earlier.

Doesn't Table already have a _repr_html_?

https://github.com/astropy/astropy/blob/master/astropy/table/table.py#L1368

Oops, yes. Thanks @mdboom, removing...

I see -- so the return trick works by virtue of the fact that you're probably running this at a console, and taking advantage of the fact that it stores a history of results. That's probably fine in IPython where it stores a lot of history (that I always find hard to get rid of), but at the standard Python console, it's not so great:

>>> t.show_in_browser()
<open file '<fdopen>', mode 'w+b' at 0x30e1c00>
>>> 42 * 3
126
>>> # refresh the browser and it's gone

Personally, I don't mind so much if "refresh" doesn't work, but I thought I'd point it out.

It also doesn't work in a script, which I think is more serious. I know it seems odd, but people may want to have scripts that generate a complicated table and then pop up the results in a browser and maybe a plot in another window.

from astropy.table import Table

def generate_and_show():
    t = Table([1,2,3])
    t.show_in_browser()

Maybe we need to delete the tempfile in a sys.atexit handler? Another hack is to stuff the tempfile in a private global variable.

Good question. One option is to state that, for persistent/script use, you need to do
w = t.show_in_browser().
In the generate_and_show example, just add a return t statement, and make that persist.

The options seem to be:

1. Document carefully, tell users to keep the NamedTemp object around
2. Use trickery and hackery to keep the object persistent until closing

I prefer 1 because it's cleaner and easier, but not strongly.

These last two commits may fall under an entirely different category... but I just added a little javascript so that the table is sortable & searchable within the browser. I could go on forever, making it nicely formatted, colored, etc, which is probably a dumb idea, but I think the sortable table is a useful feature. Note that if you run this offline, i.e. without an internet connection, it will just display the dumb pure-html table as normal.

Very cool use of javascript. Maybe it would be better to add that feature as part of _repr_html_, however, and chain to that, so that it also works in the IPython console? I don't think we want this at the lowest level of pprint, as there's also some utility to getting a really straightforward HTML table if needed.

That's a good idea; I'll split out the javascript viewer into a different function in the pprint module so that we can control the options more cleanly

@mdboom - ipython does not like the js used for this viewer, I get very odd behavior when I invoke the repr

There's the added issue with _repr_html_ that you can't pass any options, so you don't get to choose whether to include the js. If the js is included, you really want to show all columns and probably all rows. Normal repr, that's probably not the case.

Indeed. I can confirm the weirdness. Something about the initialization isn't right -- by the time the callback gets called, it doesn't get have the DOM content of the table. Probably need to hook into a different javascript callback other than $('document').ready (and the table will need a unique id, since there will be more than one of them, but that's a detail for later). Anyway -- didn't look into it much more than that. We can always move forward by reverting _repr_html_ to what it was until we can work out the IPython-specific kinks.

Right, thanks. I've decided there may be a more sensible way to do this: default repr is the same as old repr, but add a button to turn any table browseable. That deals with the callback issue.

So I got pretty close: this version creates a button that, in principle, converts the simple html table to a complete javascript viewey table.

Problems:

1. I'm not sure the ipython In[]/Out[] structures are stable enough to justify my use of jquery the way I've done it (looking for parents, then getting their children)
2. The button works, but then it doesn't: on my machine, the new JS table appears, then disappears and is replaced by undefined. I think this is a side-effect of kernel.execute?

Thanks Jake. It is a bit tricky right now, but I'm 95% of the way there, I think. In principle, the hard part is done....

...that said, can you point me to any current docs on kernel.execute, especially what exactly the callback does?

(it would be really cool to have a working demo, then move over to bigger & better things when a real API is available)

Try this in a notebook:

from astropy.table import Table
T = Table(np.ones([10,10]))
T

The Make Table Browseable button is functional, but any larger table (i.e., one whose width exceeds the screen width) will be missing information.

(use commit keflavich@a8bf772)

@mdboom Upon further consideration, I really don't think it makes sense to load the jsviewer by default with _repr_html_. It can be relatively heavy, and I think another approach could be more effective. We could make a show_in_notebook method that would look something like:

def show_in_notebook(self, *args, **etc):
    HTML(self._jsviewer() + self.pformat(html=True, max_width=np.inf, max_lines=np.inf))
    some_magical_function_to_call_the_js_function_magically()

which I think would be best, though perhaps you'd want to avoid cluttering the namespace. And I don't really know how to call the jsfunction that is added to the html via this method.

Also I've broken lots of tests and this code needs lots of tests. Perhaps the js utilities were too ambitious; those should perhaps be split into a different PR or different module entirely.

and THIS rebase seems to have worked...

That looks better. It seems like unrelated commits seem to

Sorry about the accidental close. What did you do differently the second time around? We've seen these unrelated commits happening before and it's not always clear why.

I have no idea. It might be something as simple as rebasing... I thought I had to rebase twice, but apparently not. According to bash, my command history was:

git filter-branch --force --index-filter 'git rm --cached --ignore-unmatch astropy/table/data/jquery.js' --prune-empty --tag-name-filter cat -- --all
git filter-branch --force --index-filter 'git rm --cached --ignore-unmatch astropy/table/data/jquery.dataTables.nightly.js' --prune-empty --tag-name-filter cat -- --all
git push -f origin show_in_browser
git fetch upstream
git rebase -i upstream/master
git push -f origin show_in_browser

@keflavich - it looks like there is a test failure?

@eteq @mdboom @taldcroft - does this look ready to merge? It would be cool to get in 0.3.0 so that it gets some user testing.

Looks good to me. I'd rather see the jquery stuff in a central location in case we use it outside of table in the future, but we can cross that bridge when we get there.

Fine by me.

@keflavich - good to go? or was there anything else you wanted to add?

I think it's good to go.

Looks good to go except for one minor thing: It would be nice to adjust the commit tree so that the adding of the minified js goes in the same place the origin adding of the js goes. Then if anyone for some reason wants to checkout an intermediate commit, it will still work.

That's totally minor, though, so if you're worried about the rebase on that, it's fine as-is.

So... You want more Stalinization (history editing) or just rebase commit
squashing?

@eteq - I don't think that's too important, and the issue is that the filename is different. I don't think we ever require branches to be functional until finally merged, right?

I agree with @astrofrog, as long as the end product works and there isn't too much commit noise along the way, then why bother. I think it's fair to say that most PR's cannot be checked out midway and still work.

Agreed. And for things like git bisect, all that matters is the "trunk", i.e. the merge commits.

I hadn't thought about git bisect but it's good that it does the right thing.

Ok so... shall we merge? :)

As a matter of principal, I still think it's worthwhile to have the intermediate commits work when possible - when doing more complicated merge/rebase operations, I've been burned before by this, as it's hard to tell exactly what in your merge broke the other branch's work.

But in this case, I now see @astrofrog's point that the filename is different, so it doesn't matter anyway. So I'll merge this as-is. Thanks @keflavich! (and +1 to the term "Stalinization" :)