Welcome to Scream Logbook

These are my notes from trying to make stuff work.

Sentiment Analysis

Python libraries:

Textblob
Natural Language Toolkit (NLTK)
Don't use pattern (no longer maintained)

from textblob import TextBlob

blob = TextBlob(text)

for sentence in blob.sentences:
    print(sentence.sentiment) # sentiment.polarity sentiment.subjectivity

Polarity : connotation +positive vs -negative (+happy vs -sad)

Subjectivity : +objective vs -subjective (+fact vs -opinion)

Web Requests and Scrapping

HTTP Status Codes

Curl

Curl.se

curl https://example.com

Python

import requests
r = requests.get('https://example.com/')
print(r.text)
print(r.status_code)

Beautiful Soup

Docs for Beautiful Soup 4

from bs4 import BeautifulSoup
soup = BeautifulSoup(html_doc, 'html.parser')

PHP

How to get a baisic php website running.

PHP is a Template language. It sort of has it's own html tag. <?php [code] ?>

index.php

<!DOCTYPE html>
<html>
  <body>
    <h1>Current Time:</h1>
    <?php
        $current_time = date("h:i:sa");
        echo "<p>Current time is $current_time</p>";
    ?>
  </body>
</html>

If you run php index.php > output.html & open output.html you will get an html with the current time substituted.

<!DOCTYPE html>
<html>
  <body>
    <h1>Current Time:</h1>
    <p>Current time is 04:28:45pm</p>
  </body>
</html>

In fact, it doesn't even need to be HTML.

Hello World at <?php
    $current_time = date("h:i:sa");
    echo "$current_time";
?>.

Compiles into Hello World at 04:38:02pm. just fine.

PHP just replaces it's own code with the code's output.

Server

A server that automatically compiles all requested php files is a builtin command.

php -S localhost:8000

SQLite

SQLite comes out of the box

<?php
$db = new SQLite3('sqlite3db.db');

$results = $db->query('select * from db');
while ($row = $results->fetchArray()) {
var_dump($row);
}
?>

Flask (Python)

Programming Tales

Gmail with Python

Getting into Gmail with Python.

Use a testing account.

Simple way: (the way 3rd party clients like Thunderbird do it.)

SMTP (comms between servers) smtplib is builtin
POP3 (read only from email server) poplib is builtin
IMAP (read/write with email server) imaplib is builtin
- email python library/parser

Complex way: Gmail API with Google Cloud (good luck)

Warning: Don't hardcode tokens and keys. (how to handle secrets)

Config:

List of Servers/Subdomains for Major Email Providers (imap.gmail.com)
username/email youremail@example.com
password: Special "App Password"
- You will need to click around in your account

IMAP

import os
from dotenv import load_dotenv
load_dotenv('.env')
username = os.getenv('USERNAME')
password = os.getenv('PASSWORD')

import imaplib

server = 'imap.gmail.com'
imap = imaplib.IMAP4_SSL(server)
imap.login(email, password)

print(imap.list()) # lists mailboxes
status, messages = imap.select(mailbox="INBOX",readonly=True)

Emails on the server are assinged sequential IDs. Starting from 1.

msg_uid = 10
status, msg = imap.fetch(str(msg_uid),'(RFC822)') # no clue why RFC822
# msg
#  0: msg itself
#  1: (uid, data)

import email
data = msg[1][1]
m = email.message_from_bytes(data)

print(m.get_payload())

Glossary

NOOP command (imap.noop()) is a test packet. Similar to a ping.

MIME is Multipurpose Internet Mail Extensions

['Delivered-To', 'Received', 'X-Received', 'ARC-Seal', 'ARC-Message-Signature', 'ARC-Authentication-Results', 'Return-Path', 'Received', 'Received-SPF', 'Authentication-Results', 'DKIM-Signature', 'X-Google-DKIM-Signature', 'X-Gm-Message-State', 'X-Google-Smtp-Source', 'MIME-Version', 'X-Received', 'Date', 'X-Account-Notification-Type', 'Feedback-ID', 'X-Notifications', 'X-Notifications-Bounce-Info', 'Message-ID', 'Subject', 'From', 'To', 'Content-Type']

Key	Description	Example
`Subject`	Subject line of the Email	"Shedule for Meeting"
`From`	Sender Email Address	sender-email@example.com
`To`	Recipient Email Address	recipient-email@example.com
`Date`	Date in Long Format with time zone	Sun, 19 Jan 2025 00:11:46 GMT
	More Obscure Fields
`Content-Type`	Media Type (Full List) boundary charset	`text/plain`
`Received`	by [IPv6] with [Protocol] id [alphanumberic]
`Delivered-To`	Recipient Email Address	recipient-email@example.com

Dealing with Multipart Emails

Content-Type: multipart/alternative, mutlipart/mixed, etc.

Multipart Emails contain multiple parts with different types. And many email providers/clients don't bother minimizing the type, just always use the universal multipart.

.is_multipart()

Iterator: Warning! first object is parent itself!

.walk() generator object to iteratre through all message parts.

"part" object:

.get_content_type() (get subtype)
.is_multipart() (can be nested)

Attachments

Actions

FLAGS:

Marking Read/Unread

Printing in Python

list of tricks for advanced output to the terminal

Created July 28th 2025 for Python 3.13.0

Default Print

Color

Libraries

Pretty Print

import pprint as pp

Pretty Print Plus Plus

Just better Pretty Print, though less supported.

Github (400 stars, last update in 2020)

IceCream

Print for Debugging, automatically prints the variable being printed.

from icecream import ic
ic() # will print it's own filename and line number
ic(max(3.141,6.283)) # ic| max(3.141,6.283): 6.283

Github

Rich

Powerful library for working with text in the terminal.

Docs

from rich.pretty import pprint
pprint(locals())

pip install rich

SetPrint

Python CLI Libraries and Notes

Argparse builtin
Click the classic python cli library
Typer minimalist and easy to add
Plumbum does some crazy unix stuff

Also Python Printing Libraries like [Rich]((./printing_in_python.md#rich) will be really relevant.

Multiple Git Identities / Profiles

Multiple Git Profiles Guide by Icaruk

Dealing with Secrets

Never ever hardcode snsitive information, especially when publishing it. Put whatever holds the secrets into .gitignore.

TOC:

dotenv (Python)

Dotenv

L main.py
L .env

PASSWORD=password1234

import os # necessary
from dotenv import load_dotenv

load_dotenv('.env') # Loads into OS

password = os.getenv('PASSWORD') # Gets from the OS

(os.getenv("HOME") is Bash environmental variables $ env)

YAML

Android Termux

Default package manager is pkg

But it doesn't work for 99% of stuff, so expect to compile and git clone stuff yourself. Also, superuser is broken unless you have rooted your phone.

SSH server

SSH = Secure Shell (encrypted connection to another computers commandline)

Official docs on Remote Access.

Doing everything from the computer is 10x easier, so use ssh.

In Termux

Install OpenSSH pkg install openssh
Host with sshd (opens on port 8022)
1. This exposes your device only on the local (WiFi) network.
2. Check your ip address in ifconfig. 127.0.0.1 is localhost, instead you want the 192.168.[...].[...] local address.
whoami should give current user (something like u0_a123)
Setup password with passwd (Remember step 1.1, so don't set defaults)
Do whatever you wanted to do.

Last. pkill sshd to close

Hosting stuff

Only real usecase of using a phone for selfhosting, is when you don't have your laptop, but other people for some reason do, and you want your phone to do the heavy lifting.

Figure out IP you will be hosting stuff from:

Localhost: 127.0.0.1 or localhost.
Local (home network): ifconfig (192.168.[...].[...])
Public:

Awesome Selfhosted

What I managed to get running:

Atheos Web IDE

Github

Normal install won't work because Termux doesn't like sudo.

Atheos is written in PHP. So

Hence git clone https://github.com/Atheos/Atheos

cd Atheos

php -S [your local ip]:8000 (local network php server on port 8000)

Open [phone's local ip]:8000 in your computer browser and enjoy.

JupyterLab Python Notebook

Github

Given how Termux doesn't like pip, and how JupyterLab + Kernels are generally hard to get working, messing with JupyterLab will likely be futile.

Try pip3 install jupyterlab at pray that it works.

VS Code

Github

You can do VS Code online at vscode.dev, only caviat, you need to pay Github for hosting, or provide your own server.

At this point, consider signing up for Replit or some other online IDE. But if you are really set on hosting from your phone: they conviniently have a tutorial on their official website. (tldr pkg install code-server)

No JupyterLab because pip breaks
No Marimo because pip breaks

Python

Yes, pip constantly fails to install stuff

Compiling C Projects

Read the damn docs and compilation instructions if available.

Classic

Classic way of compiling almost any project written in C.

./configure && make && make install

Breakdown:

./configure executes configure file to check dependencies

! Note that configure often doesn't exist initially, and a autogen.sh (or similar) must be run (bash autogen.sh) to generate configure and other setup stup steps.

make

make install

Docs

Instructions for compiling/building/running are usually in README.md, INSTALLING.md, or a similar file. cat to read them.

The docs for small projects are usually outdated, files might have been moved around within the source code.

Use tree command to see what files there are.

Search for mentioned files in the source code via fuzzy finder fzf or find . -name "autogen*" (find command options starting/path Regex)

C Libraries

pkg-config --list-all to see all libraries you have installed.

If you need to install a library, you can compile it yourself using pretty much the same steps here.

LittleJS

LittleJS Github Repository

curl https://raw.githubusercontent.com/KilledByAPixel/LittleJS/refs/heads/main/dist/littlejs.js > littlejs.js

<!DOCTYPE html><head>
<title>LittleJS Hello World Demo</title>
<meta charset=utf-8>
</head><body>

<script src=littlejs.js></script>
<script src=game.js></script>

curl https://raw.githubusercontent.com/KilledByAPixel/LittleJS/refs/heads/main/examples/empty/game.js > game.js

Lynx Browser

Famously cool browser, confusingly difficult to run.

Official Lynx Website has the latest stable version listed, as of writing it is 2.9.2.

Installing

Package Manager

Lynx is fairly popular, so most package managers have a lynx package. (see here)

You can check the version with lynx --version

Compiling it yourself

w3m

w3m is a CLI web browser written by a Japanese guy.

It is hosted on SouceForge.

Like most other C project, run ./configure && make && make install in the soure code directory.

Elinks

https://github.com/rkd77/elinks download latest release from releases (as of writing v0.18.0)

./configure && make && make install

you might notice ./configure doesn't work

Run ./autogen.sh and it configure will appear.

.configure should produce a list of features that will and won't work based on prerequisite libraries. (Even if some are missing, that should be ok.)

make should compile and list bunch of linking (LD and CC) for the above libraries.

make install should put the final binary into your bin folder.

elinks should work now

Installing Libraries

"tre" Regex Library

Browsh

Browsh Offical Website and Browsh Github repository, also Browsh

As of writting, browsh is version 1.8.3. (browsh --version)

Browsh is actually fairly well documented. See SETUP_DEV_[Platform].md for compilation instructions.x

To Quit Browsh: Ctrl+Q

Context on innerworkings: Browsh has Firefox running in the background, and mirrors it into ANSI text on the terminal. It thus requires Firefox installed, and relies on it. In fact, you can view the mirrored instance using browsh --firefox.with-gui.

OPTical ARchiver (optar)

If you are building optar yourself, download from github (git clone https://github.com/colindean/optar)

Outdated Website

Dependencies

Libpng (see compiling libpng)
ImageMagick

Compiling

make will compile everything

make install (sudo make install if necessary) will install optar, uoptar, and pgm2ps into bin. (/usr/local/bin) you can check that with which [command].

pgm2ps

pgm2ps is actually a shell script. You can open it in a plaintext editor and edit it.

Run which pgm2ps, and edit it using any editor. (eg. vi $(which pgm2ps))

solve outdated imagemagic warning (see below)
change from A4 to US Letter formats (see below)

Writing (Encoding)

optar <input file> [filename base] (filename base is basically output file name without an extension)

This will roduce a filename-base.pgm file. PGM is a "grayscale image format".

Converting pgm2ps Stage

pgm2ps <file.pgm> or pgm2ps *.pgm to convert ALL .pgm files in the directory into printable PostScript (.ps).

If there is a warning about magick and covert. Run which pgm2ps, (pgm2ps is just a shell script) and vi <path>. Then replace covert command with magick.

A4 and US Letter changes

If you want to change to A4 (default) or US Letter, you have to edit which pgm2ps file. Substitute the following equations as described in comments within pgm2ps:

US Letter: 573.84x755.28+19.08+18.36
A4: 556.56x807.12+19.22+17.44

PostScript (.ps) is the file format printers use for printing. Gimp can open .ps and .pgm file formats, for checking if they are ok.

Reading (Decoding)

Just check the docs

You will need an industrial grade scanner for this.

unoptar <format-specs> filename-base > outputfile

Dragon

https://github.com/mwh/dragon

How to run the dragon cli tool for drag and dropping files.

Dependencies: GTK+ 3, X11 or Wayland

You are required to build it yourself, no binaries are available. Also development seems to have halted with v1.2.0 in 2022. These instructions are for MacOS.

The official README has great installation instructions.

To start, cd into some folder you don't mind having a bit of junk files in. eg. cd ~/Downloads

git clone https://github.com/mwh/dragon.git
cd dragon to enter the folder with the sourcecode
make will use the Makefile file to compile dragon binary.
1. Dealing with dependecy issues. Dragon requries GTK+ 3. For MacOS brew install gtk+3
2. You might need to install X or X11 window manager. Just google how to install it.
make install to but the dragon binary into the bin folder.
dragon will run dragon and print Usage: dragon [OPTIONS] FILENAME
dragon example will create a window with a drag-and-droppable folder from the example folder in the source code directory. (ls)

Tre Regex Library

https://github.com/laurikari/tre

git clone https://github.com/laurikari/tre

./utils/autogen.sh (creates configure executeable among other things)

./configure
make
make check
make install

Libpng

Download source

Warning, the configure and other files have carriage retrun \r characters. You need to remove them, because sh hates them.

To remove carriage returns

sed -i.bak 's/\r$//g' <filename>

sed -i.bak 's/\r$//g' configure

./configure
make check
make install

Currently still can't get it to work.

grab-site website archiver

Github repo by Archive Team

Installation on Apple Silicon

(on August 2025 for MacOS 15.6 with Python 3.13.0 and grab-site 2.2.7) brew update update homebrew brew install libxslt re2 pkg-config install libraries via homebrew

libxslt is for XML parsing
re2 is a regex library
pkg-config is for compiling libraries (not sure) python3 -m venv ~/gs-venv create python virtual environment, and puts its config at ~/gs-venv PKG_CONFIG_PATH="/usr/local/opt/libxml2/lib/pkgconfig" make sure pkg-config works before the next step ~/gs-venv/bin/pip3 install lxml install lxml library into virtual environment ~/gs-venv/bin/pip3 install git+https://github.com/ArchiveTeam/grab-site install grab-site from github into the virtual environment. If it fails to compile due to cchardet being deprecated.

Add PATH="$PATH:$HOME/gs-venv/bin" to ~/.zshrc (or equivalent). Then restart terminal or source .zshrc.

Usage

gs-server and see menu at http://127.0.0.1:29000/ (localhost)

grab-site 'URL'

Windows Operating System

Entering boot menu:

Shift + Restart
Troubleshooting
Advanced options
Start
Fn + F12

Windows 11 Debloat

Windows 10 Debloat

Python GUI Libraries

DearPyGui

working with File System in Python

Main libraries:

import os (os docs)
import pathlib (pathlib docs)

Setup

For pathlib operations of file system, Path("the/path/string.txt")
For pathlib operations of file path use PurePath("the/path/string.txt")

R/W on known paths

Check if stuff even exists

os.path.exists(file_path_os) (check if path exists)
os.path.isfile(file_path_os) check if thing is a file (not dir)
Path.exists()
- Path.mkdir(parents=True)

Parsing

os.path.split()
path = pathlib.PurePath("path/goes/here.txt") Pathlib accessing individual components
- PurePath.anchor (root baisically)
- PurePath.name -> filename.txt
- PurePath.suffix -> .txt
- PurePath.stem -> filename without suffix

HTML Boilerplate

Sources:

HTMLBoilerplates.com

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8" />
        <title>Template Title</title>
        <meta name="viewport" content="width=device-width,initial-scale=1" />
        <meta name="description" content="Description Goes Here" />
        <link rel="stylesheet" type="text/css" href="style.css" />
    </head>
    <body>
        <h1>Template</h1>
        <script src="index.js"></script>
    </body>
</html>

Fullscreen

// On pressing ENTER call toggleFullScreen method
document.addEventListener("keydown", (e) => {
  if (e.key === "Enter") {
    toggleFullScreen(element);
  }
});
function toggleFullScreen(element) {
  if (!document.fullscreenElement) {
    // If the document is not in full screen mode
    // make the video full screen
    element.requestFullscreen();
  } else {
    // Otherwise exit the full screen
    document.exitFullscreen?.();
  }
}

Canvas API

#! /usr/bin/env python3

chmod +x script.sh

Ruby

written June 2025 on MacOS 15.5

~~brew install ruby~~

MacOS has version 2.6 ruby preinstalled (which is outdated, current>3.0). There is nothing you can do about this.

The brew installation won't work, because the 2.6 binary takes priority over the 3.0, so nothing will work out of the box.

Stackoverflow thread on this issue

Check paths with type -a ruby or which -a ruby

Check versions with ruby --version, gem --version

Which is why you need Ruby Environment Manager:

Ruby Environment Manager

rbenv

brew install rbenv ruby-build

rbenv install -l

rbenv install <version>

just follow Stackoverflow thread on this issue, it's to confusing and complicated to explain what is acutally going on.

RubyGems

RubyGems.org Website

The gem package manager comes preinstalled with ruby.

gem update --system to update gem

ERROR:  While executing gem ... (Gem::FilePermissionError)
    You don't have write permissions for the /Library/Ruby/Gems/2.6.0 directory.

MacOS is defaulting to outdated 2.6 ruby. See above.

README Style Guide

link to repo somewhere at the top. Finding the source / remote origin is hard in cloned repositories.

Art of README
Software Relsease Pracieces by The Linux Documentation Project
GNU Releases

Profiling .zshrc

If your terminal takes a long time to show the first prompt, it is because sourcing your .zshrc is slow.

.zshrc is a shell script that runs every time you open a new terminal. It is a hidden file in the home folder: ~/.zshrc It is used by package managers and other shell tools to configure themselves before hand. Consequently, you will see a lot of conda, nvm, etc. in there.

Run source is the equivalent of import. So source ~/.zshrc will re-load your shell.

Trivia: The "rc" stands for "RUNCOM" according to wikipedia Wikipedia. Other shells like bash also have .bashrc files.

How to Profile

Time stamped echos:

echo "Start $(date +%s%N)ns" # nanoseconds, 9 places after the second
# ... suspects go here ...
echo "end   $(date +%s%N)ns"

time [utility] is usefull for timing the execution time of a single function. time sh test.sh with a suspect line in test.sh will print the execution time.

To print a nice table with breakdown of time consuming functions use (scroll to the top of the output):

zmodload zsh/zprof
# zshrc here
zprof

Warning: zprof is wierd and actually adds up the number of calls if you run source ~/.zshrc again in the same terminal session.

Comment out the thing that is taking up time, and see how fast the startup is. Then decide whether to remove it or speed it up or keep it.

# source $ZSH/oh-my-zsh.sh # 90% of my time

What to look for

source means a whole additional script is being run, which could have plenty of time consuming stuff hidden inside. Btw. . is an alias for source so if you see a . "script.sh", make sure to go after it.

Package managers. Whereever there is a package manager, there will be source and other time consuming stuff.

Install lazy loading plugin from here. It uses lazy loading (loads only once needed.)
conda initialization is infamously slow. See this reddit comment for a lazy loader
nvm is a common suspect. Google how to lazy load it as well.

Terminal customizers, such as oh-my-zsh. They are the main consumers of loading time, and and supposed to be. Look into tweaking them or using a different one if they are to slow. Google possible solutions here is one for oh-my-zsh.

From the Profiler output

Look at the top 10 lines of the zprof output.

compdump and compinit and compaudit are shell completions (when you press tab and it show you available commands). They are the the heaviest suspect. They run only once, and remove them if possible. I have found them being run multiple times on multiple occasions.

Open the .zshrc file in a text editor and Ctrl+F for compinit and just the word completions. Even if compinit is not in the file, package managers often attach comments.
Not all completions are guilty. For example [ -s "/Users/me/.bun/_bun" ] && source "/Users/me/.bun/_bun" (bun's completion) doesn't call compinit and is very fast when measured with time or echo "$(date +%s%N)"

What Not to Look For

export is fast. Unlike source, export doesn't execute anything.

Powerlevel10k is famously fast.

More Resources

https://www.mikekasberg.com/blog/2025/05/29/optimizing-zsh-init-with-zprof.html
https://github.com/romkatv/zsh-bench?tab=readme-ov-file#summary
https://www.dribin.org/dave/blog/archives/2024/01/01/zsh-performance/
https://kasad.com/blog/zsh-profiling/
https://alextheobold.com/posts/profiling_zsh/