Skip to content

[FIX] wery long execution on fatty apps#1548

Closed
kvokka wants to merge 5 commits intopry:masterfrom
kvokka:patch-1
Closed

[FIX] wery long execution on fatty apps#1548
kvokka wants to merge 5 commits intopry:masterfrom
kvokka:patch-1

Conversation

@kvokka
Copy link
Copy Markdown

@kvokka kvokka commented Jun 19, 2016

When ObjectSpase become to be huge, autocomplete with double tab press beginning to work weeery slow (up to 20 seconds for act), and main core of it is going throw ObjectSpace & sort vs uniq after (at the end I have 40k records). I made hack, which will do it only if count of object changes, so, if you debug, you'll need to wait only once ( sad, but it is inevitably), and then you'll use saved collection.

Also I improved selection function, so it will work faster too. so you'll wait ~3 second first time and then it will work instantly.

I can not test in jruby, in MRI all works perfect.

Also, may me you'll find useful this idea like an option with external updater for methods and put this option as a settings. If you like this Idea, i can rewrite this chunk.

kvokka added 2 commits June 19, 2016 14:35
When ObjectSpase become to be huge, autocomplete with double tab beginning to work weeery slow (up to 20 seconds for act), and main core of it is going throw ObjectSpace & sort vs uniq after (at the end I have 40k records). I made hack, which will do it only if count of object changes, so, if you debug, you'll need to wait only once ( sad, but it is  inevitably), and then you'll use saved collection.

Also I improved selection function, so it will work faster too.
JRuby support (will be tested with Travis)
Comment thread lib/pry/input_completer.rb Outdated
WORD_ESCAPE_STR = " \t\n\"\\'`><=;|&{("

class << self
attr_accessor :obj_space_collection, :obj_space_count, :obj_spase_mutex
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why obj_spase_mutex instead of obj_space_mutex ?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ooops + copy/paste = spase :)

self.obj_space_collection = buffer.keys.map(&:to_s).sort
end.count
end
end
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jazzonmymind Thank you! I've added .count, it was my mistake.

@0x1eef
Copy link
Copy Markdown
Contributor

0x1eef commented Jul 6, 2016

overall this looks okay, but the error on jruby as well as the typo will have to be fixed before it can be merged.

@kvokka
Copy link
Copy Markdown
Author

kvokka commented Jul 7, 2016

I rewrite the method and now it is not bad.
Also it shows up a memory leak on autocomplete, but it is another story.
I made a stub for it for now, so, all works.

For re-execute Travis
@kvokka
Copy link
Copy Markdown
Author

kvokka commented Jul 17, 2016

As you can see my code do not crush MRI, and I suggest, that it may be a jruby tests problem, is based not on my code.

If I'm wrong and you have other Ideas I'll be happy to hear it (because this patch really make life easier on big projects)


private

def self.fetch_obj_space_collection
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kvokka wdyt on separating mutex logic, checking for stale cache and preparing methods list? I hope it'll be easier to read and understand.

def all_available_methods_cached
  @mutext.sync do
    @all_available_methods_cached = all_available_methods if stale_methods_cache?
    @all_available_methods_cached
  end
end

def stale_methods_cache?
  # check that modules count changed
end

def all_available_methods
  # Fetch from ObjectSpace
end

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And if you already have class << self section on the top, it would be better to move this methods there.

Mutext can be also set once with a constant: MUTEXT = Mutex.new

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@printercu There is no problem to refactor it, but as I see, this code was written more than 3 month ago. Now, unfortunately, I see, that this chunk is still unuseful for repo owner. If it helps somebody- awesome!

You may take this snippet and make own PW with refactored code, if you wish.

About singleton in the bottom, you, probably, missed, that it is in private section. I prefer put all private methods in 1 section.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kvokka Ok, I'll take a look. BTW, private modifier doesn't effect class-methods: http://rubocop.readthedocs.io/en/latest/cops_lint/#lintineffectiveaccessmodifier

Or one-liner class A; private; def self.must_be_private; end; end; A.public_methods(false)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@printercu Thank you! This is good sample.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jazzonmymind I'm almost ready with similar to this PR refactoring, I'll make PR soon. I think method cache should be global, because it doesn't depend on pry's state.

Copy link
Copy Markdown
Contributor

@0x1eef 0x1eef Nov 13, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't understand that argument.
an instance of Pry::InputCompleter already belongs to an instance of Pry, we just re-initialise it every time Pry accepts input. Avoiding global state is always a good thing, and the only reason it has to be global is because we throw away the instance of Pry::InputCompleter at each eval tick. we could initialise one instance of Pry::InputCompleter, store the method cache on that instance, and avoid the lock altogether. locks should be a last resort, and global state should be avoided where possible. in this case, sharing the cache between all Pry::InputCompleter's has no benefit as far as i can tell, and the cache could fill up with methods that don't relate between Pry instances. it's in fact a negative imo.

Copy link
Copy Markdown

@printercu printercu Nov 13, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. On the other hand completer instance is connected to binding, and we would need to update it on every switch. IMO This would not be good design. I sew that binding is an arg of #call

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you mean by "on every switch" ? the binding is received in Pry::InputCompleter#call, and that wouldn't change. #initialize accepts an input object(like Readline for example) and an instance of Pry.

the only difference between global and instance-local state would be that we would store the collection and count as instance variables on an instance of Pry::InputCompleter, and avoid the mutex and class-level state entirely. nothing else would change, even how bindings are passed into Pry::InputCompleter would not change.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I see. Sorry fro my comment before I've checked the code. I've added a not in #1583 (comment)

@0x1eef
Copy link
Copy Markdown
Contributor

0x1eef commented Nov 13, 2016

closed in favour of #1583.

@0x1eef 0x1eef closed this Nov 13, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants