Welcome to Scream Logbook
These are my notes from trying to make stuff work.
My setup is MacOS with Apple Selicone.
Sentiment Analysis
Python libraries:
- Textblob
- Natural Language Toolkit (NLTK)
- Don't use pattern (no longer maintained)
from textblob import TextBlob
blob = TextBlob(text)
for sentence in blob.sentences:
print(sentence.sentiment) # sentiment.polarity sentiment.subjectivity
Polarity : connotation +positive vs -negative (+happy vs -sad)
Subjectivity : +objective vs -subjective (+fact vs -opinion)
More materials
-
Word Lists and Sentiment Analysis by Neal Caren with AFINN (python library)
-
Sentiment Analysis Word Lists Dataset by Prajwal Kanade on Kaggle
Web Requests and Scrapping
Curl
curl https://example.com
Python
import requests
r = requests.get('https://example.com/')
print(r.text)
print(r.status_code)
Beautiful Soup
from bs4 import BeautifulSoup
soup = BeautifulSoup(html_doc, 'html.parser')
PHP
How to get a baisic php website running.
PHP is a Template language. It sort of has it's own html tag. <?php [code] ?>
index.php
<!DOCTYPE html>
<html>
<body>
<h1>Current Time:</h1>
<?php
$current_time = date("h:i:sa");
echo "<p>Current time is $current_time</p>";
?>
</body>
</html>
If you run php index.php > output.html & open output.html
you will get an html with the current time substituted.
<!DOCTYPE html>
<html>
<body>
<h1>Current Time:</h1>
<p>Current time is 04:28:45pm</p>
</body>
</html>
In fact, it doesn't even need to be HTML.
Hello World at <?php
$current_time = date("h:i:sa");
echo "$current_time";
?>.
Compiles into Hello World at 04:38:02pm.
just fine.
PHP just replaces it's own code with the code's output.
Server
A server that automatically compiles all requested php files is a builtin command.
php -S localhost:8000
SQLite
<?php
$db = new SQLite3('sqlite3db.db');
$results = $db->query('select * from db');
while ($row = $results->fetchArray()) {
var_dump($row);
}
?>
Flask (Python)
Programming Tales
Gmail with Python
Getting into Gmail with Python.
Use a testing account.
Simple way: (the way 3rd party clients like Thunderbird do it.)
- SMTP (comms between servers)
smtplib
is builtin - POP3 (read only from email server)
poplib
is builtin - IMAP (read/write with email server)
imaplib
is builtinemail
python library/parser
Complex way: Gmail API with Google Cloud (good luck)
Warning: Don't hardcode tokens and keys. (how to handle secrets)
Config:
- List of Servers/Subdomains for Major Email Providers (
imap.gmail.com
) - username/email
youremail@example.com
- password: Special "App Password"
- You will need to click around in your account
IMAP
import os
from dotenv import load_dotenv
load_dotenv('.env')
username = os.getenv('USERNAME')
password = os.getenv('PASSWORD')
import imaplib
server = 'imap.gmail.com'
imap = imaplib.IMAP4_SSL(server)
imap.login(email, password)
print(imap.list()) # lists mailboxes
status, messages = imap.select(mailbox="INBOX",readonly=True)
Emails on the server are assinged sequential IDs. Starting from 1.
msg_uid = 10
status, msg = imap.fetch(str(msg_uid),'(RFC822)') # no clue why RFC822
# msg
# 0: msg itself
# 1: (uid, data)
import email
data = msg[1][1]
m = email.message_from_bytes(data)
print(m.get_payload())
Glossary
NOOP
command (imap.noop()
) is a test packet. Similar to a ping.
MIME is Multipurpose Internet Mail Extensions
Header Fields
['Delivered-To', 'Received', 'X-Received', 'ARC-Seal', 'ARC-Message-Signature', 'ARC-Authentication-Results', 'Return-Path', 'Received', 'Received-SPF', 'Authentication-Results', 'DKIM-Signature', 'X-Google-DKIM-Signature', 'X-Gm-Message-State', 'X-Google-Smtp-Source', 'MIME-Version', 'X-Received', 'Date', 'X-Account-Notification-Type', 'Feedback-ID', 'X-Notifications', 'X-Notifications-Bounce-Info', 'Message-ID', 'Subject', 'From', 'To', 'Content-Type']
Key | Description | Example |
---|---|---|
Subject | Subject line of the Email | "Shedule for Meeting" |
From | Sender Email Address | sender-email@example.com |
To | Recipient Email Address | recipient-email@example.com |
Date | Date in Long Format with time zone | Sun, 19 Jan 2025 00:11:46 GMT |
More Obscure Fields | ||
Content-Type | Media Type (Full List) boundary charset | text/plain |
Received | by [IPv6] with [Protocol] id [alphanumberic] | |
Delivered-To | Recipient Email Address | recipient-email@example.com |
Dealing with Multipart Emails
Content-Type: multipart/alternative
, mutlipart/mixed
, etc.
Multipart Emails contain multiple parts with different types. And many email providers/clients don't bother minimizing the type, just always use the universal multipart.
.is_multipart()
Iterator: Warning! first object is parent itself!
.walk()
generator object to iteratre through all message parts.
"part" object:
.get_content_type()
(get subtype).is_multipart()
(can be nested)
Attachments
Actions
FLAGS:
Printing in Python
list of tricks for advanced output to the terminal
Created July 28th 2025 for Python 3.13.0
Default Print
Color
Libraries
Pretty Print
import pprint as pp
Pretty Print Plus Plus
Just better Pretty Print, though less supported.
Github (400 stars, last update in 2020)
IceCream
Print for Debugging, automatically prints the variable being printed.
from icecream import ic
ic() # will print it's own filename and line number
ic(max(3.141,6.283)) # ic| max(3.141,6.283): 6.283
Rich
Powerful library for working with text in the terminal.
from rich.pretty import pprint
pprint(locals())
pip install rich
SetPrint
Python CLI Libraries and Notes
- Argparse builtin
- Click the classic python cli library
- Typer minimalist and easy to add
- Plumbum does some crazy unix stuff
Also Python Printing Libraries like [Rich]((./printing_in_python.md#rich) will be really relevant.
Multiple Git Identities / Profiles
Multiple Git Profiles Guide by Icaruk
Dealing with Secrets
Never ever hardcode snsitive information, especially when publishing it. Put whatever holds the secrets into .gitignore
.
TOC:
Dotenv
L main.py
L .env
PASSWORD=password1234
import os # necessary
from dotenv import load_dotenv
load_dotenv('.env') # Loads into OS
password = os.getenv('PASSWORD') # Gets from the OS
(os.getenv("HOME")
is Bash environmental variables $ env
)
YAML
Android Termux
Default package manager is pkg
But it doesn't work for 99% of stuff, so expect to compile and git clone stuff yourself. Also, superuser is broken unless you have rooted your phone.
SSH server
SSH = Secure Shell (encrypted connection to another computers commandline)
Official docs on Remote Access.
Doing everything from the computer is 10x easier, so use ssh.
In Termux
-
Install OpenSSH
pkg install openssh
-
Host with
sshd
(opens on port 8022)- This exposes your device only on the local (WiFi) network.
- Check your ip address in
ifconfig
.127.0.0.1
islocalhost
, instead you want the192.168.[...].[...]
local address.
-
whoami
should give current user (something likeu0_a123
) -
Setup password with
passwd
(Remember step 1.1, so don't set defaults) -
Do whatever you wanted to do.
Last. pkill sshd
to close
Hosting stuff
Only real usecase of using a phone for selfhosting, is when you don't have your laptop, but other people for some reason do, and you want your phone to do the heavy lifting.
Figure out IP you will be hosting stuff from:
- Localhost:
127.0.0.1
orlocalhost
. - Local (home network):
ifconfig
(192.168.[...].[...]
) - Public:
What I managed to get running:
Atheos Web IDE
Normal install won't work because Termux doesn't like sudo.
Atheos is written in PHP. So
Hence git clone https://github.com/Atheos/Atheos
cd Atheos
php -S [your local ip]:8000
(local network php server on port 8000)
Open [phone's local ip]:8000 in your computer browser and enjoy.
JupyterLab Python Notebook
Given how Termux doesn't like pip, and how JupyterLab + Kernels are generally hard to get working, messing with JupyterLab will likely be futile.
Try pip3 install jupyterlab
at pray that it works.
VS Code
You can do VS Code online at vscode.dev, only caviat, you need to pay Github for hosting, or provide your own server.
At this point, consider signing up for Replit or some other online IDE. But if you are really set on hosting from your phone: they conviniently have a tutorial on their official website. (tldr pkg install code-server
)
- No JupyterLab because pip breaks
- No Marimo because pip breaks
Python
Yes, pip constantly fails to install stuff
Compiling C Projects
- Read the damn docs and compilation instructions if available.
Classic
Classic way of compiling almost any project written in C.
./configure && make && make install
Breakdown:
./configure
executes configure
file to check dependencies
! Note that configure
often doesn't exist initially, and a autogen.sh
(or similar) must be run (bash autogen.sh
) to generate configure
and other setup stup steps.
make
make install
Docs
Instructions for compiling/building/running are usually in README.md
, INSTALLING.md
, or a similar file. cat
to read them.
The docs for small projects are usually outdated, files might have been moved around within the source code.
Use tree
command to see what files there are.
Search for mentioned files in the source code via fuzzy finder fzf
or find . -name "autogen*"
(find command options starting/path Regex
)
C Libraries
pkg-config --list-all
to see all libraries you have installed.
If you need to install a library, you can compile it yourself using pretty much the same steps here.
LittleJS
curl https://raw.githubusercontent.com/KilledByAPixel/LittleJS/refs/heads/main/dist/littlejs.js > littlejs.js
<!DOCTYPE html><head>
<title>LittleJS Hello World Demo</title>
<meta charset=utf-8>
</head><body>
<script src=littlejs.js></script>
<script src=game.js></script>
curl https://raw.githubusercontent.com/KilledByAPixel/LittleJS/refs/heads/main/examples/empty/game.js > game.js
js13k build and build.js
Download the build.js
script from the LittleJS repo, and chmod +x build.js
to give it execution permissions. (Or just run it with node
)
build.js
won't work out of the box, go in and edit the variables at the top.
Issues:
- Some
npx <tool>
fails to run. Look it up on npmjs.com and install globally vianpm
. eg. I had issues with Uglify JS. - If the
game.zip
is not appearing, chech ifect
(efficient compression tool) is installed (which ect
). see my notes on compilingect
Lynx Browser
Famously cool browser, confusingly difficult to run.
Official Lynx Website has the latest stable version listed, as of writing it is 2.9.2.
Installing
Package Manager
Lynx is fairly popular, so most package managers have a lynx package. (see here)
You can check the version with lynx --version
Compiling it yourself
w3m
w3m is a CLI web browser written by a Japanese guy.
It is hosted on SouceForge.
Like most other C project, run ./configure && make && make install
in the soure code directory.
Elinks
https://github.com/rkd77/elinks download latest release from releases (as of writing v0.18.0)
./configure && make && make install
you might notice ./configure
doesn't work
Run ./autogen.sh
and it configure
will appear.
.configure
should produce a list of features that will and won't work based on prerequisite libraries. (Even if some are missing, that should be ok.)
make
should compile and list bunch of linking (LD and CC) for the above libraries.
make install
should put the final binary into your bin
folder.
elinks
should work now
Installing Libraries
Browsh
Browsh Offical Website and Browsh Github repository, also Browsh
As of writting, browsh is version 1.8.3. (browsh --version
)
Browsh is actually fairly well documented. See SETUP_DEV_[Platform].md
for compilation instructions.x
To Quit Browsh: Ctrl+Q
Context on innerworkings: Browsh has Firefox running in the background, and mirrors it into ANSI text on the terminal. It thus requires Firefox installed, and relies on it. In fact, you can view the mirrored instance using browsh --firefox.with-gui
.
OPTical ARchiver (optar)
If you are building optar yourself, download from github (git clone https://github.com/colindean/optar
)
Dependencies
- Libpng (see compiling libpng)
- ImageMagick
Compiling
make
will compile everything
make install
(sudo make install
if necessary) will install optar
, uoptar
, and pgm2ps
into bin. (/usr/local/bin
) you can check that with which [command]
.
pgm2ps
pgm2ps
is actually a shell script. You can open it in a plaintext editor and edit it.
Run which pgm2ps
, and edit it using any editor. (eg. vi $(which pgm2ps)
)
- solve outdated imagemagic warning (see below)
- change from A4 to US Letter formats (see below)
Writing (Encoding)
optar <input file> [filename base]
(filename base is basically output file name without an extension)
This will roduce a filename-base.pgm
file. PGM is a "grayscale image format".
Converting pgm2ps Stage
pgm2ps <file.pgm>
or pgm2ps *.pgm
to convert ALL .pgm
files in the directory into printable PostScript (.ps
).
If there is a warning about magick
and covert
. Run which pgm2ps
, (pgm2ps is just a shell script) and vi <path>
. Then replace covert
command with magick
.
A4 and US Letter changes
If you want to change to A4 (default) or US Letter, you have to edit which pgm2ps
file. Substitute the following equations as described in comments within pgm2ps:
- US Letter: 573.84x755.28+19.08+18.36
- A4: 556.56x807.12+19.22+17.44
PostScript (.ps
) is the file format printers use for printing. Gimp can open .ps
and .pgm
file formats, for checking if they are ok.
Reading (Decoding)
Just check the docs
You will need an industrial grade scanner for this.
unoptar <format-specs> filename-base > outputfile
Dragon
https://github.com/mwh/dragon
How to run the dragon cli tool for drag and dropping files.
Dependencies: GTK+ 3, X11 or Wayland
You are required to build it yourself, no binaries are available. Also development seems to have halted with v1.2.0 in 2022. These instructions are for MacOS.
The official README has great installation instructions.
To start, cd
into some folder you don't mind having a bit of junk files in. eg. cd ~/Downloads
-
git clone https://github.com/mwh/dragon.git
-
cd dragon
to enter the folder with the sourcecode -
make
will use theMakefile
file to compile dragon binary.- Dealing with dependecy issues. Dragon requries GTK+ 3. For MacOS
brew install gtk+3
- You might need to install X or X11 window manager. Just google how to install it.
- Dealing with dependecy issues. Dragon requries GTK+ 3. For MacOS
-
make install
to but the dragon binary into the bin folder. -
dragon
will run dragon and printUsage: dragon [OPTIONS] FILENAME
-
dragon example
will create a window with a drag-and-droppable folder from theexample
folder in the source code directory. (ls
)
Tre Regex Library
https://github.com/laurikari/tre
git clone https://github.com/laurikari/tre
./utils/autogen.sh
(creates configure
executeable among other things)
./configure
make
make check
make install
Libpng
Warning, the configure
and other files have carriage retrun \r
characters. You need to remove them, because sh hates them.
To remove carriage returns
sed -i.bak 's/\r$//g' <filename>
sed -i.bak 's/\r$//g' configure
./configure
make check
make install
ect
Currently still can't get it to work.
grab-site website archiver
Github repo by Archive Team
Installation on Apple Silicon
(on August 2025 for MacOS 15.6 with Python 3.13.0 and grab-site 2.2.7)
brew update
update homebrew
brew install libxslt re2 pkg-config
install libraries via homebrew
- libxslt is for XML parsing
- re2 is a regex library
- pkg-config is for compiling libraries (not sure)
python3 -m venv ~/gs-venv
create python virtual environment, and puts its config at~/gs-venv
PKG_CONFIG_PATH="/usr/local/opt/libxml2/lib/pkgconfig"
make sure pkg-config works before the next step~/gs-venv/bin/pip3 install lxml
install lxml library into virtual environment~/gs-venv/bin/pip3 install git+https://github.com/ArchiveTeam/grab-site
install grab-site from github into the virtual environment. If it fails to compile due to cchardet being deprecated.
Add PATH="$PATH:$HOME/gs-venv/bin"
to ~/.zshrc
(or equivalent). Then restart terminal or source .zshrc
.
Usage
gs-server
and see menu at http://127.0.0.1:29000/ (localhost)
grab-site 'URL'
Windows Operating System
Entering boot menu:
- Shift + Restart
- Troubleshooting
- Advanced options
- Start
- Fn + F12
Python GUI Libraries
DearPyGui
working with File System in Python
Main libraries:
import os
(os docs)import pathlib
(pathlib docs)
Setup
- For pathlib operations of file system,
Path("the/path/string.txt")
- For pathlib operations of file path use
PurePath("the/path/string.txt")
R/W on known paths
Check if stuff even exists
os.path.exists(file_path_os)
(check if path exists)os.path.isfile(file_path_os)
check if thing is a file (not dir)Path.exists()
Path.mkdir(parents=True)
Parsing
os.path.split()
path = pathlib.PurePath("path/goes/here.txt")
Pathlib accessing individual componentsPurePath.anchor
(root baisically)PurePath.name
->filename.txt
PurePath.suffix
->.txt
PurePath.stem
->filename
without suffix
HTML Boilerplate
Sources:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<title>Template Title</title>
<meta name="viewport" content="width=device-width,initial-scale=1" />
<meta name="description" content="Description Goes Here" />
<link rel="stylesheet" type="text/css" href="style.css" />
</head>
<body>
<h1>Template</h1>
<script src="index.js"></script>
</body>
</html>
// On pressing ENTER call toggleFullScreen method
document.addEventListener("keydown", (e) => {
if (e.key === "Enter") {
toggleFullScreen(element);
}
});
function toggleFullScreen(element) {
if (!document.fullscreenElement) {
// If the document is not in full screen mode
// make the video full screen
element.requestFullscreen();
} else {
// Otherwise exit the full screen
document.exitFullscreen?.();
}
}
Related
#! /usr/bin/env python3
chmod +x script.sh
Ruby
written June 2025 on MacOS 15.5
brew install ruby
MacOS has version 2.6 ruby preinstalled (which is outdated, current>3.0). There is nothing you can do about this.
The brew installation won't work, because the 2.6 binary takes priority over the 3.0, so nothing will work out of the box.
Stackoverflow thread on this issue
Check paths with type -a ruby
or which -a ruby
Check versions with ruby --version
, gem --version
Which is why you need Ruby Environment Manager:
Ruby Environment Manager
rbenv
brew install rbenv ruby-build
rbenv install -l
rbenv install <version>
just follow Stackoverflow thread on this issue, it's to confusing and complicated to explain what is acutally going on.
RubyGems
The gem
package manager comes preinstalled with ruby.
gem update --system
to update gem
ERROR: While executing gem ... (Gem::FilePermissionError)
You don't have write permissions for the /Library/Ruby/Gems/2.6.0 directory.
MacOS is defaulting to outdated 2.6 ruby. See above.
README Style Guide
- link to repo somewhere at the top. Finding the source / remote origin is hard in cloned repositories.
Related:
- Art of README
- Software Relsease Pracieces by The Linux Documentation Project
- GNU Releases
Profiling .zshrc
If your terminal takes a long time to show the first prompt, it is because sourcing your .zshrc
is slow.
.zshrc
is a shell script that runs every time you open a new terminal. It is a hidden file in the home folder: ~/.zshrc
It is used by package managers and other shell tools to configure themselves before hand. Consequently, you will see a lot of conda
, nvm
, etc. in there.
Run source
is the equivalent of import. So source ~/.zshrc
will re-load your shell.
Trivia: The "rc" stands for "RUNCOM" according to wikipedia Wikipedia. Other shells like bash also have .bashrc
files.
How to Profile
Time stamped echos:
echo "Start $(date +%s%N)ns" # nanoseconds, 9 places after the second
# ... suspects go here ...
echo "end $(date +%s%N)ns"
time [utility]
is usefull for timing the execution time of a single function.
time sh test.sh
with a suspect line in test.sh will print the execution time.
To print a nice table with breakdown of time consuming functions use (scroll to the top of the output):
zmodload zsh/zprof
# zshrc here
zprof
Warning: zprof is wierd and actually adds up the number of calls if you run source ~/.zshrc
again in the same terminal session.
Comment out the thing that is taking up time, and see how fast the startup is. Then decide whether to remove it or speed it up or keep it.
# source $ZSH/oh-my-zsh.sh # 90% of my time
What to look for
source
means a whole additional script is being run, which could have plenty of time consuming stuff hidden inside. Btw. .
is an alias for source
so if you see a . "script.sh"
, make sure to go after it.
Package managers. Whereever there is a package manager, there will be source
and other time consuming stuff.
- Install lazy loading plugin from here. It uses lazy loading (loads only once needed.)
conda
initialization is infamously slow. See this reddit comment for a lazy loadernvm
is a common suspect. Google how to lazy load it as well.
Terminal customizers, such as oh-my-zsh
. They are the main consumers of loading time, and and supposed to be. Look into tweaking them or using a different one if they are to slow. Google possible solutions here is one for oh-my-zsh.
From the Profiler output
Look at the top 10 lines of the zprof output.
compdump
and compinit
and compaudit
are shell completions (when you press tab and it show you available commands). They are the the heaviest suspect. They run only once, and remove them if possible. I have found them being run multiple times on multiple occasions.
- Open the
.zshrc
file in a text editor andCtrl+F
forcompinit
and just the wordcompletions
. Even ifcompinit
is not in the file, package managers often attach comments. - Not all completions are guilty. For example
[ -s "/Users/me/.bun/_bun" ] && source "/Users/me/.bun/_bun"
(bun's completion) doesn't call compinit and is very fast when measured withtime
orecho "$(date +%s%N)"
What Not to Look For
export
is fast. Unlike source
, export
doesn't execute anything.
Powerlevel10k is famously fast.
More Resources
- https://www.mikekasberg.com/blog/2025/05/29/optimizing-zsh-init-with-zprof.html
- https://github.com/romkatv/zsh-bench?tab=readme-ov-file#summary
- https://www.dribin.org/dave/blog/archives/2024/01/01/zsh-performance/
- https://kasad.com/blog/zsh-profiling/
- https://alextheobold.com/posts/profiling_zsh/