Skip to content

michaelbrusegard/lektr

Repository files navigation

Lektr Compendium Website

WEBSITE LINK Since we are not allowed to use Docker we could not run the S3 service on the VM (Minio S3 requires Docker). Therefor file uploads will not work in the editor, but you can still embed files via URL. BACKUP LINK If you are trying to create an account on the website, you will get stuck on the verifyEmail page. To get past this you can use "42424242" as the verification code. We have not implemented sending emails yet as this is out of scope for the delivery. The account you create will not be trusted so you can not use it for editing.

PS! If you get back a response with "too many requests" when using the website, it is because of the global rate limiter used to prevent abuse. The rate limiting bucket will be filled up again shortly so it should work again in 2-3 minutes. It is per IP address so if you switch to another IP address it will also work again. We tried increasing the rate limit for deliverable 3, but if it is not high enough please let us know.

We really wanted to make a website that we felt could be of use for others. One group member had a friend who has created compendiums for some courses on NTNU and he wanted to share them. Based on this we wanted to make a website for compendiums similar to https://kateter.no and https://www.enkeleksamen.no, but it will be free. The friend is really into charity so he wanted there to be a way to donate to charity for students who like his work. The goal is to make the experience as good as possible for reading the compendiums, that is why we want to try to make it possible to have built in exercises in the compendiums. We also want dark mode and mobile support for a better user experience. Below is a list of features we have in mind, but we are not sure if we will implement them all.

  • Dark mode: Better reading experience with dark background
  • Warm Aesthetic: We want to make it possible to have a warm and friendly paperlike feeling on the website to make it more appealing to read.
  • Mobile support
  • Built in exercises: We want to make it possible to have built in exercises in the compendiums to help learning.
  • Authentication: We want to make it possible to have a login system (Maybe for tracking donations or a comment system, not decided yet)
  • Internationalization: We want to make it possible to have the website in different languages so international students can use it too.
  • Charity donation: We want to make it possible to have a way to donate to charity.
  • ChatGPT integration: This will probably not happen since it costs money but it would be useful to have a chat interface with ChatGPT that has a context of the compendium.
  • Image and video support: We want to make it possible to have images and videos in the compendiums that are uploaded to our own server.
  • Easy way to create a compendium: We want to make it possible to have a easy way to create a compendium using a block based editor.

Now these are just some ideas, we are not sure if we will implement all of them.

Deliverables

Here are some reflections on the deliverables:

How to run

To run the project, make sure you have your favorite Package Manager and JavaScript runtime installed. We use Bun (because it's fast), but you can use Node and npm as well. Start by installing the dependencies:

bun install # or npm install

Then you need to create a .env file in the root directory of the project and fill it with environment variables. You can use the .env.example file as a template.

Note

You need to have a Postgres database running on your local machine or connect to one on a server. There is provided a docker-compose file in the root directory of the project that you can use to start a local Postgres database.

Then run the development server:

bun dev # or npm run dev

This will start a development server on localhost:3000/project2.

To build the project for production, run:

bun run build # or npm run build

And then start it with:

bun run start # or npm run start

Running Backend

After starting the production build, or dev server the backend will already be running. It is exposed on the /api route on the web server. The reason for this is that we embed the backend in Next.js instead of running it as a separate process. It will behave just the same way, but since Next.js is already running on the server, it would be a bit of a waste to run two server processes.

Warning

Trying to build the application on the VM will not work and it will most likely crash.

Database setup

To setup the database make sure you first have docker installed and running. You can start the database with the following command:

bun run db:start

This will start a local Postgres database on the specified port in the .env file using docker-compose. You then need to migrate the database to create the tables:

bun run db:migrate

You can then seed the database with some data:

bun run db:seed

Now everything should be set up and the website should work with data. For file uploads to work you need to setup and host a Minio S3 server or connect to AWS S3. For the Minio S3 server you can use the similar commands to the database setup provided in the package.json file.

NTNU Virtual Machine

These are the commands to setup the project on the NTNU virtual machine. First the project needs to be cloned from git and you need to be in the root directory of the project on the VM.

# Install bun
curl -fsSL https://bun.sh/install | bash

# Install dependencies
bun run install

# Install Postgres
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo apt-get update
sudo apt-get -y install postgresql

# Install pm2
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt-get install -y nodejs
sudo npm install pm2 -g

# Run pm2 on startup
pm2 startup
pm2 save

# Start Postgres service
sudo systemctl start postgresql
sudo systemctl enable postgresql

# Create database and user using the correct values from the .env file
sudo -u postgres psql -c "CREATE USER \"LJbfvM9gfcM=\" WITH PASSWORD 'vuVHlnRN1WDl+KSDXCVdKg==';"
sudo -u postgres psql -c "CREATE DB \"lektr-db\";"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DB \"lektr-db\" TO \"LJbfvM9gfcM=\";"
sudo -u postgres psql -d "lektr-db" -c "GRANT ALL ON SCHEMA public TO \"LJbfvM9gfcM=\";"


# Configure Postgre to allow connections
sudo sed -i "s/#listen_addresses = 'localhost'/listen_addresses = '*'/" /etc/postgresql/*/main/postgresql.conf
echo "host    all             all             0.0.0.0/0               md5" | sudo tee -a /etc/postgresql/*/main/pg_hba.conf

# Restart Postgres service
sudo systemctl restart postgresql

Then create a .env file in the root directory of the project and fill it with these values:

# General
NODE_OPTIONS="--max_old_space_size=16384"
NEXT_TELEMETRY_DISABLED="true"
NEXT_PUBLIC_SITE_URL="http://it2810-27.idi.ntnu.no"
NEXT_PUBLIC_CONTACT_EMAIL="contact@email.com"

# Database
DB_HOST="localhost"
DB_PORT="5432"
DB_USER="LJbfvM9gfcM="
DB_PASSWORD="vuVHlnRN1WDl+KSDXCVdKg=="
DB_NAME="lektr-db"

# Storage
S3_HOST="localhost"
S3_PORT="9000"
S3_USER="aP8IYWT1n7A="
S3_PASSWORD="vlJNGsgRMDvjV5OoWFIwfg=="
S3_BUCKETS="editor"

# Services
OPENAI_ORGANIZATION=" "
OPENAI_PROJECT=" "
OPENAI_API_KEY=" "

Removed the OPENAI values since the should be kept private.

Then on your local machine, using the same .env file you need to run the build. This is because the VM is not powerful enough to build the project.

# Make sure to have bun installed
# Install dependencies
bun install

# Run build
bun run build

# Copy the build to the VM
scp -r .next/* YOUR_USERNAME@it2810-27.idi.ntnu.no:~/T27-Project-2/.next/

Back on the VM, you can now start the project with pm2.

pm2 start "bun run start" --name "T27-Project-2"

Accessibility

Our project prioritizes web accessibility through several key implementations:

  • Semantic HTML: We use proper semantic elements throughout the application to ensure screen readers can properly interpret content structure.
  • ARIA Labels: All interactive elements have appropriate ARIA labels and roles where needed.
  • Keyboard Navigation: The entire application is fully navigable via keyboard, with visible focus indicators.
  • Color Contrast: We maintain WCAG 2.1 compliant color contrast ratios for all text and interactive elements.
  • Responsive Text: Font sizes use relative units and scale appropriately across different screen sizes.
  • Alt Text: All images have descriptive alt text, and decorative images are properly marked.
  • Form Accessibility: Form inputs have associated labels and error messages are announced by screen readers.
  • Radix UI: We use Radix UI primitives which provide built-in accessibility features including:
    • Proper focus management
    • ARIA attributes
    • Keyboard interactions
    • Screen reader announcements
  • Dark Mode: Reduces eye strain by providing a dark theme option.
  • Responsive Design: Content is accessible across all device sizes and orientations.

Sustainability

We've implemented several sustainable web development practices:

  • Server-Side Rendering: Using Next.js SSR reduces client-side processing and improves energy efficiency by:

    • Sending pre-rendered HTML instead of large JavaScript bundles
    • Reducing client CPU usage for initial renders
    • Enabling better caching strategies
  • Performance Optimizations:

    • Image optimization with Next.js built-in image optimization
    • Code splitting and lazy loading of components
    • Efficient bundle size management
    • Caching strategies for API responses
    • Minimized JavaScript execution
  • Resource Efficiency:

    • CSS-first approach using Tailwind to reduce JavaScript overhead
    • Optimized database queries to reduce server load
    • Rate limiting to prevent resource abuse
    • Efficient state management with Zustand
    • GraphQL to prevent over-fetching of data
  • JSON Patch Implementation: We significantly reduce network bandwidth by sending only changed data:

    • Instead of full document updates, we send minimal JSON patches
    • Typically 90-95% smaller payload sizes compared to full document transfers
    • Example: Updating a single word sends ~20 bytes instead of the entire document
  • Hosting and Infrastructure:

    • Consolidated backend and frontend on same server to reduce network overhead
    • Database connection pooling
    • Efficient file storage using S3
    • Proper caching headers for static assets

These practices help reduce the carbon footprint of our application by minimizing unnecessary processing, network traffic, and server resources while maintaining excellent user experience.

SSR and Next.js

We are using Next.js and SSR to optimize the performance of our project. This document highlights the differences between our setup and a typical Vite setup. We've divided the explanation into two parts: a brief overview of the differences, and a detailed explanation for those interested in the specifics.

Note

This document does not cover differences arising from our use of specific dependencies. For information on a specific tool, refer to its official documentation.

Any feedback with questions related to SSR is welcome.

What is different?

  • Some components have "use client" at the top. This can be ignored.
  • Instead of a index.html file like in Vite, we define the entry in src/app/[locale]/layout.tsx. The head element is populated using methods provided by Next.js like generateMetadata and Viewport. This allows dynamic meta tags per page, improving SEO.
  • Routing is handled by Next.js during compilation, not specified in a separate file like with react-router-dom. This is done using file-based routing where the name of a directory indicates the route. For example, app/about/page.tsx will be accessible at /project2/about.
    • layout.tsx files act like Higher-Order Components (HOC) in React, wrapping the page component. They encapsulate each subsequent layout and page component in the directory hierarchy.
    • loading.tsx files are similar to Suspense in React. They wrap the page component in a Suspense that is shown while the page is loading.
    • The not-found.tsx and error.tsx files are shown when a page is not found or an error occurs.
    • A directory with square brackets (app/[locale]) indicates a dynamic route. For example, the locale can be en or no in the URL.
    • A directory with parentheses (app/[locale]/(default)) is used for structure and will be ignored from the URL. It is useful if you want to have a different layout for pages with the same root path.

SSR explained

  • With Next.js, every page is rendered on the server by default. Instead of downloading an empty HTML file with a script tag that includes React, the user receives a fully rendered HTML file. This allows the page to be cached and the user can start using the site immediately without waiting for the JavaScript to load and render the page.
  • The "use client" directive tells Next.js that the component should also be rendered on the client. This is important for user interactions. Once the first "use client" is encountered, all subsequent components are also rendered on the client. To be able to use client specific code like React hooks and DOM interactions, the component has to be rendered on the client.
  • The advantage of SSR comes down to performance and SEO. A page rendered on the server is more SEO-friendly as search crawlers see a full HTML file, not just an HTML with a script tag that includes React. Downloading a prerendered HTML file is faster than downloading the whole React bundle.

Dependencies

Frontend

  • React - Library for building user interfaces
  • Next.js - Framework for routing and server-side rendering
  • Next-intl - Internationalization library
  • next-themes - Dark mode library with built in context and updating of local storage
  • Tanstack Query - Data fetching library with caching and with loading states
  • Tanstack Table - For dynamic tables with filtering, sorting, pagination etc
  • Tanstack Form - When we need to handle form validation
  • Zustand - State management library
  • tsparticles - Cool particles library we can use as backgrounds
  • Slate - Rich text editor
  • Plate - Building blocks for slate to create your own custom UI
  • Radix UI Primitives - Unstyled component library we are using for our UI components with built-in accessibility and responsiveness
  • shadcn/ui - Copy paste styles on top of Radix UI Primitives

Styling

Backend

  • Drizzle - ORM for interacting with the database (Postgres under the hood)
  • Postgres.js - Postgres client
  • s3-client - AWS S3 client for uploading files
  • GraphQL Yoga - GraphQL server for our API
  • gql.tada - GraphQL Typescript inferrer
  • superjson - JSON library for storing and retrieving data
  • zod - TypeScript schema validation
  • fast-json-patch - Patching library for JSON
  • The Copenhagen Book - Resource for learning ins and outs of setting up secure session authentication

Other resources

  • Mozilla - Great resource for looking up documentation for web technologies
  • Can I use - Check browser support for different web technologies (especially useful for CSS)

Requirements Checklist

Functionality

  • The solution should be a prototype of a searchable catalog with a frontend where the user can formulate a search and be presented with a search result, which the user can subsequently interact with.
  • Search functionality with a dialog/form/search field for input of search.
  • List-based presentation of search results with handling of large result sets either by paging through pages, or dynamically loading more results when scrolling.
  • The ability to see more details about each of the objects.
  • Ability to sort and filter the result set (note that sorting and filtering should be performed on the entire result set and not just what happens to be loaded on the client).
  • There should be some form of user-generated data that should be stored (persistently on the database server) and presented (either user adding information, reviews, ratings etc, history about searches or other, shopping list).
  • The solution should demonstrate aspects of universal design / web accessibility (accessibility).
  • The solution should demonstrate aspects of sustainable web development (through choices made in design).
  • Good design, sensible choices and solutions that harmonize with the type of data you choose.
  • Database and backend for the project should be hosted on the group's virtual machine at submission.

Use of Technology

  • User interface should be based on React and programmed in TypeScript. The project should be set up with Vite.
  • Use of state management for example redux, mobx, recoil, apollo local state management etc.
  • Custom/developed GraphQL backend (Node / TypeScript), free choice of type of database server on backend, but the project should use a backend database set up by the group.
  • Use of good and relevant components and libraries (free choice and we encourage as much reuse of third-party solutions as possible).

Testing, Development and Quality Control

  • Linting and use of Prettier.
  • Conducted testing of components (we use Vitest).
  • Some form of automated end-2-end testing (in practice testing a longer sequence of interactions), testing of the API.
  • The project is documented with a README.md in the git repository. The documentation should discuss, explain and refer to all the most important choices and solutions that the group makes (including choice of components and api).
  • The code should be easy to read and well structured and commented so that it is easy to get into. Use of comments should be adapted to external inspection of the code.
  • The group should summarize each person's contribution to the project along the way in a separate file that is delivered in BB (this is personal information that no one wants to be on git ;-)
  • Reproducibility: in practice this means that the project should be documented and easy to install/run for others (for example the teacher).

Intermediate Submissions

  • First intermediate submission should only be a first version of the system with mockup data or statically coded data etc. Should show what data you have, how the application is intended to be designed in terms of data display, interaction with the result etc. The goal of the submission is to get started, show off your own idea and get feedback, see what others have planned. No strict requirements other than that fellow students should be able to clone the repo and look at the app locally minimum.
  • Second intermediate submission should have a running backend and support interaction in the form of searching, filtering, sorting and paging. Here the focus is on a functioning interaction between different functionality.
  • Third intermediate submission should in practice be a nearly finished project, but with the possibility of being changed during or after peer assessment with regard to the final assessment. Here we assess all aspects of the application including testing, accessibility requirements and sustainability perspective.
  • What is listed are minimum requirements and it is of course possible to deliver something that has come further in development for each intermediate delivery.

Setup on Virtual Machine

  • The project is made available via Apache web server on the group's vm under the address http://it2810-xx.idi.ntnu.no/project2
  • Backend should preferably be available via port 3001
  • All information necessary to start up the backend should at least be available to the academic staff (e.g. documented on git, or in a file that can be read with sudo rights on vm)

Final Evaluation

Application Understanding

  • Shows good insight in design and functionality for this type of applications.
  • Chooses good solutions, tries out and explores functionality rather than choosing simple.
  • Has focus on user-friendliness and good user experience.

Technical Understanding

  • Has solution that will scale and work regardless of data volume.
  • The application is thoroughly tested for typical and atypical use.
  • Has chosen good and effective solutions in implementation and development, including with regard to sustainability.
  • Demonstrates development competence at all levels.

Design and Formulation

  • Has an excellent design in terms of visual and interactive.
  • Shows very good understanding of and has implemented focus on accessibility.

About

A web application with an advanced text editor built with Web Sockets. It is currently a WiP and not open sourced. It is built to be a tool for students to collaborate and share resources for courses at university.

Resources

Stars

Watchers

Forks

Contributors

Languages