Skip to content

avoid using pkg_resources for faster command execution#426

Merged
jleveque merged 1 commit intosonic-net:masterfrom
ishidawataru:fastentry
Jan 11, 2019
Merged

avoid using pkg_resources for faster command execution#426
jleveque merged 1 commit intosonic-net:masterfrom
ishidawataru:fastentry

Conversation

@ishidawataru
Copy link
Copy Markdown
Contributor

@ishidawataru ishidawataru commented Jan 5, 2019

- What I did

speed up command execution by avoiding 'pkg_resources' import.

related issue : #408

- How I did it

Used https://github.com/ninjaaron/fast-entry_points

- How to verify it

- Previous command output (if the output of a command-line utility has changed)

admin@sonic:~$ time show
Usage: show [OPTIONS] COMMAND [ARGS]...

  SONiC command line - 'show' command

Options:
  -?, -h, --help  Show this message and exit.

Commands:
  aaa                   Show AAA configuration
  acl                   Show ACL related information
  arp                   Show IP ARP table
  clock                 Show date and time
  ecn                   Show ECN configuration
  environment           Show environmentals (voltages, fans, temps)                                                                                                                                                                                                                                                                                                         interfaces            Show details of the network interfaces
  ip                    Show IP (IPv4) commands
  ipv6                  Show IPv6 commands
  line                  Show all /dev/ttyUSB lines and their info
  lldp                  LLDP (Link Layer Discovery Protocol)...
  logging               Show system log
  mac                   Show MAC (FDB) entries
  mirror_session        Show existing everflow sessions
  mmu                   Show mmu configuration
  ndp                   Show IPv6 Neighbour table
  ntp                   Show NTP information
  pfc                   Show details of the priority-flow-control...
  platform              Show platform-specific hardware info                                                                                                                                                                                                                                                                                                                priority-group        Show details of the PGs
  processes             Display process information
  queue                 Show details of the queues
  reboot-cause          Show cause of most recent reboot
  runningconfiguration  Show current running configuration...
  services              Show all daemon services
  startupconfiguration  Show startup configuration information
  system-memory         Show memory information
  tacacs                Show TACACS+ configuration
  techsupport           Gather information for troubleshooting
  uptime                Show system uptime
  users                 Show users
  version               Show version information
  vlan                  Show VLAN information
  warm_restart          Show warm restart configuration and state
  watermark             Show details of watermark

real    0m1.376s
user    0m1.188s
sys     0m0.164s

- New command output (if the output of a command-line utility has changed)

admin@sonic:~$ time show
Usage: show [OPTIONS] COMMAND [ARGS]...

  SONiC command line - 'show' command

Options:
  -?, -h, --help  Show this message and exit.

Commands:
  aaa                   Show AAA configuration
  acl                   Show ACL related information
  arp                   Show IP ARP table
  clock                 Show date and time
  ecn                   Show ECN configuration
  environment           Show environmentals (voltages, fans, temps)
  interfaces            Show details of the network interfaces
  ip                    Show IP (IPv4) commands
  ipv6                  Show IPv6 commands                                                                                                                                                                                                                                                                                                                                  line                  Show all /dev/ttyUSB lines and their info
  lldp                  LLDP (Link Layer Discovery Protocol)...
  logging               Show system log
  mac                   Show MAC (FDB) entries
  mirror_session        Show existing everflow sessions
  mmu                   Show mmu configuration
  ndp                   Show IPv6 Neighbour table
  ntp                   Show NTP information
  pfc                   Show details of the priority-flow-control...
  platform              Show platform-specific hardware info
  priority-group        Show details of the PGs
  processes             Display process information
  queue                 Show details of the queues                                                                                                                                                                                                                                                                                                                          reboot-cause          Show cause of most recent reboot
  runningconfiguration  Show current running configuration...
  services              Show all daemon services
  startupconfiguration  Show startup configuration information
  system-memory         Show memory information
  tacacs                Show TACACS+ configuration
  techsupport           Gather information for troubleshooting
  uptime                Show system uptime
  users                 Show users
  version               Show version information
  vlan                  Show VLAN information
  warm_restart          Show warm restart configuration and state
  watermark             Show details of watermark

real    0m0.760s
user    0m0.608s
sys     0m0.140s

avoid using pkg_resources for faster command execution
e6c2c7a 
Signed-off-by: Wataru Ishida <ishida@nel-america.com>
@jleveque
Copy link
Copy Markdown
Contributor

jleveque commented Jan 7, 2019

@ishidawataru: Thanks for looking into this issue. Does this change also speed up tab-completion?

@ishidawataru
Copy link
Copy Markdown
Contributor Author

ishidawataru commented Jan 7, 2019

@jleveque Yes it should. Every tab-completion executes the command to complete internally.

@jleveque jleveque merged commit da3934e into sonic-net:master Jan 11, 2019
@ishidawataru ishidawataru deleted the fastentry branch January 11, 2019 02:10
yxieca added a commit that referenced this pull request Jan 16, 2019
@yxieca
Revert "avoid using pkg_resources for faster command execution (#426)"
0b98b54 
This reverts commit da3934e.
@jipanyang
Copy link
Copy Markdown
Contributor

jipanyang commented Jan 21, 2019

This commit is causing deb build failure:

sonic-buildimage$ make target/debs/python-sonic-utilities_1.2-1_all.deb
+++ --- Making target/debs/python-sonic-utilities_1.2-1_all.deb --- +++
BLDENV=stretch make -f Makefile.work stretch
make[1]: Entering directory '/home/jipan/warm_reboot/sonic-buildimage'
SONiC Build System

Build Configuration
"CONFIGURED_PLATFORM"             : "vs"
"SONIC_CONFIG_PRINT_DEPENDENCIES" : ""
"SONIC_BUILD_JOBS"                : "8"
"SONIC_CONFIG_MAKE_JOBS"          : "8"
"USERNAME"                        : "admin"
"PASSWORD"                        : "admin"
"ENABLE_DHCP_GRAPH_SERVICE"       : ""
"SHUTDOWN_BGP_ON_START"           : ""
"ENABLE_PFCWD_ON_START"           : ""
"INSTALL_DEBUG_TOOLS"             : ""
"ROUTING_STACK"                   : "quagga"
"ENABLE_SYNCD_RPC"                : ""
"ENABLE_ORGANIZATION_EXTENSIONS"  : "y"
"HTTP_PROXY"                      : ""
"HTTPS_PROXY"                     : ""
"ENABLE_SYSTEM_TELEMETRY"         : "y"
"SONIC_DEBUGGING_ON"              : "y"
"SONIC_PROFILING_ON"              : ""
"KERNEL_PROCURE_METHOD"           : "build"
"BUILD_TIMESTAMP"                 : "20190121.194906"

make: Nothing to be done for 'stretch'.
make[1]: Leaving directory '/home/jipan/warm_reboot/sonic-buildimage'
make -f Makefile.work target/debs/python-sonic-utilities_1.2-1_all.deb
make[1]: Entering directory '/home/jipan/warm_reboot/sonic-buildimage'
SONiC Build System

Build Configuration
"CONFIGURED_PLATFORM"             : "vs"
"SONIC_CONFIG_PRINT_DEPENDENCIES" : ""
"SONIC_BUILD_JOBS"                : "8"
"SONIC_CONFIG_MAKE_JOBS"          : "8"
"USERNAME"                        : "admin"
"PASSWORD"                        : "admin"
"ENABLE_DHCP_GRAPH_SERVICE"       : ""
"SHUTDOWN_BGP_ON_START"           : ""
"ENABLE_PFCWD_ON_START"           : ""
"INSTALL_DEBUG_TOOLS"             : ""
"ROUTING_STACK"                   : "quagga"
"ENABLE_SYNCD_RPC"                : ""
"ENABLE_ORGANIZATION_EXTENSIONS"  : "y"
"HTTP_PROXY"                      : ""
"HTTPS_PROXY"                     : ""
"ENABLE_SYSTEM_TELEMETRY"         : "y"
"SONIC_DEBUGGING_ON"              : "y"
"SONIC_PROFILING_ON"              : ""
"KERNEL_PROCURE_METHOD"           : "build"
"BUILD_TIMESTAMP"                 : "20190121.194906"

[ FAIL LOG START ] [ target/debs/python-sonic-utilities_1.2-1_all.deb ]
/sonic/src/sonic-utilities /sonic
error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/test-easy-install-156.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /usr/local/lib/python2.7/dist-packages/

Perhaps your account does not have write access to this directory?  If the
installation directory is a system-owned directory, you may need to sign in
as the administrator or "root" account.  If you do not have administrative
access to this machine, you may wish to choose a different installation
directory, preferably one that is listed in your PYTHONPATH environment
variable.

For information on other options, you may wish to consult the
documentation at:

  https://pythonhosted.org/setuptools/easy_install.html

Please make the appropriate changes for your system and try again.

[  FAIL LOG END  ] [ target/debs/python-sonic-utilities_1.2-1_all.deb ]
slave.mk:315: recipe for target 'target/debs/python-sonic-utilities_1.2-1_all.deb' failed
make: *** [target/debs/python-sonic-utilities_1.2-1_all.deb] Error 1
Makefile.work:127: recipe for target 'target/debs/python-sonic-utilities_1.2-1_all.deb' failed
make[1]: *** [target/debs/python-sonic-utilities_1.2-1_all.deb] Error 2
make[1]: Leaving directory '/home/jipan/warm_reboot/sonic-buildimage'
Makefile:6: recipe for target 'target/debs/python-sonic-utilities_1.2-1_all.deb' failed
make: *** [target/debs/python-sonic-utilities_1.2-1_all.deb] Error 2

But if I go to previous commit, no problem seen. Does the commit require new build from scratch including slave build docker?

    
jipan@jipan_sonic_vm_150:~/warm_reboot/sonic-buildimage/src/sonic-utilities$ git checkout da3934ec0b06d5a7f49785a62f13a798695f6509
Previous HEAD position was f7d46ed Revert "[reboot] stop docker service before rebooting (#423)" (#434)
HEAD is now at da3934e avoid using pkg_resources for faster command execution (#426)
jipan@jipan_sonic_vm_150:~/warm_reboot/sonic-buildimage/src/sonic-utilities$ 
jipan@jipan_sonic_vm_150:~/warm_reboot/sonic-buildimage/src/sonic-utilities$ git checkout 220e269e8334a89db5a2f931dea90e238feae7cc
Previous HEAD position was da3934e avoid using pkg_resources for faster command execution (#426)
HEAD is now at 220e269 [reboot] stop docker service before rebooting (#423)

@jipanyang jipanyang mentioned this pull request Jan 21, 2019
Closed
@ishidawataru ishidawataru mentioned this pull request Jan 23, 2019
Merged
@jleveque
Copy link
Copy Markdown
Contributor

jleveque commented Jan 31, 2019 •
edited
Loading

@ishidawataru: This change doesn't appear to speed up bash completion at all. Typing show and pressing tab twice hangs for 2.651 seconds before displaying results. Have you noticed any improvement with bash completion, or with anything else, for that matter?

I am currently building from the head of the master branch of sonic-buildimage (currently commit sonic-net/sonic-buildimage@2b01beb), and I am still seeing from pkg_resources import load_entry_point being called.

@ishidawataru ishidawataru mentioned this pull request Feb 8, 2019
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@jleveque jleveque jleveque approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

Enhancement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ishidawataru @jleveque @jipanyang @stcheng