A program designed to both "scrape" UCSD's WebReg for enrollment data and make the WebReg API available to other applications.

By default, this binary will constantly make requests to WebReg to get the number of students enrolled or waitlisted in each class, along with the total number of seats for each section. These requests are made through your account, and the session cookies are obtained through a helper webregautoin script.

This binary also includes a web server that automatically allows external applications to make requests to various WebReg endpoints; for example, external applications can make requests to the scraper's web server (which will forward that request to WebReg) to get information like course information, prerequisites, and so on. Additionally, external applications can make requests to said web server to perform actions like enrolling in classes, dropping classes, and so on.

Note that the web server, by default, is unprotected. This means that anyone can make requests to it without needing to provide authorization. This is good enough if you intend on using the web server locally only. However, if you want to make your web server accessible to the public, you can use the binary's auth feature, which sets up a very simple authorization system where external applications must provide a bearer token when making a request to the server. To manage bearer tokens for the server, or to see what authorization looks like, use the authmanager binary (or read more about it there).

Note: Starting with v0.5.0, the web server (including all WebReg endpoints) will be bundled with the scraper. This design choice was intentional. In previous versions of the binary, a web server has always been included (although, depending on the executable type, it could be minimal or feature-packed). This has always required a bit of extra maintenance on my part, so I've pretty much decided to just include everything going forward. If you want to use the scraper but do not want the web server, you'll need to manually modify the codebase to remove it.

Below are instructions on how you can run this project. Please ensure that you've set the webregautoin script up, and that it's running, before continuing.

1. Download the relevant executable here. The executables will be in an archive whose name is of the form webreg-[target]-[features]. Here,

   • [target] is the target operating system that you can run the executable for. At this time, we have
     • x86_64-apple-darwin: MacOS
     • x86_64-pc-windows-msvc: Windows
     • x86_64-unknown-linux-gnu: Linux
   • [features] is the features included in this binary. At this time, we have
     • default: Only the scraper and web server are included.
     • auth: The scraper and web server (which includes basic authentication) are included.

2. A sample configuration file has been provided for you; this file is called config.example.json.

   1. Rename this file to config.json.
   2. Modify the configuration information appropriately. Information on the structure of the configuration file can be found below.
   3. Save your changes.

3. Extract the executable from the archive.

4. Run the executable like so,

   ./webreg <path_to_config_file>
   

   where <path_to_config_file> is the name of your configuration file (assuming it's in the same directory as the executable).

If you're interested in self-compiling, follow the instructions below.

1. You'll need to install Rust. If you're on Ubuntu (or a similar system), you'll need to install the following system dependencies:

   sudo apt-get update
   sudo apt install build-essential
   sudo apt install pkg-config
   sudo apt install libssl-dev
   

2. Clone, or download, this repository.

3. You can now compile the project.

   If you'd like both the scraper and web server, without authentication, run the command

   cargo build --release --bin webreg
   

   If you'd like the scraper and web server with basic authentication, run the command

   cargo build --release --bin webreg --features auth
   

4. You should find the webreg executable in the /target/release directory. Under the "Using Pre-Compiled Executable" section, follow step 2 to set your configuration file up, and step 4 to run the executable.

In order to run this binary, you'll need to provide a configuration file. Below, you'll get an idea of what the configuration file should look like. All entries are required. For an example of this configuration file, check out config.example.json.

All information below will be in the root object.

All entries below are under apiBaseEndpoint.

All entries below are under wrapperData.

All entries below are under wrapperData[n].searchQuery, where n is some integer used to index the array.

I'll only focus on the program's main feature -- tracking enrollment counts.