Welcome to Scream Logbook

These are my notes from trying to make stuff work.

Sentiment Analysis

Python libraries:

Textblob
Natural Language Toolkit (NLTK)
Don't use pattern (no longer maintained)

from textblob import TextBlob

blob = TextBlob(text)

for sentence in blob.sentences:
    print(sentence.sentiment) # sentiment.polarity sentiment.subjectivity

Polarity : connotation +positive vs -negative (+happy vs -sad)

Subjectivity : +objective vs -subjective (+fact vs -opinion)

Web Requests and Scrapping

HTTP Status Codes

Curl

Curl.se

curl https://example.com

Python

import requests
r = requests.get('https://example.com/')
print(r.text)
print(r.status_code)

Beautiful Soup

Docs for Beautiful Soup 4

from bs4 import BeautifulSoup
soup = BeautifulSoup(html_doc, 'html.parser')

PHP

How to get a baisic php website running.

PHP is a Template language. It sort of has it's own html tag. <?php [code] ?>

index.php

<!DOCTYPE html>
<html>
  <body>
    <h1>Current Time:</h1>
    <?php
        $current_time = date("h:i:sa");
        echo "<p>Current time is $current_time</p>";
    ?>
  </body>
</html>

If you run php index.php > output.html & open output.html you will get an html with the current time substituted.

<!DOCTYPE html>
<html>
  <body>
    <h1>Current Time:</h1>
    <p>Current time is 04:28:45pm</p>
  </body>
</html>

In fact, it doesn't even need to be HTML.

Hello World at <?php
    $current_time = date("h:i:sa");
    echo "$current_time";
?>.

Compiles into Hello World at 04:38:02pm. just fine.

PHP just replaces it's own code with the code's output.

Server

A server that automatically compiles all requested php files is a builtin command.

php -S localhost:8000

SQLite

SQLite comes out of the box

<?php
$db = new SQLite3('sqlite3db.db');

$results = $db->query('select * from db');
while ($row = $results->fetchArray()) {
var_dump($row);
}
?>

Programming Tales

Gmail with Python

Getting into Gmail with Python.

Use a testing account.

Simple way: (the way 3rd party clients like Thunderbird do it.)

SMTP (comms between servers) smtplib is builtin
POP3 (read only from email server) poplib is builtin
IMAP (read/write with email server) imaplib is builtin
- email python library/parser

Complex way: Gmail API with Google Cloud (good luck)

Warning: Don't hardcode tokens and keys. (how to handle secrets)

Config:

List of Servers/Subdomains for Major Email Providers (imap.gmail.com)
username/email youremail@example.com
password: Special "App Password"
- You will need to click around in your account

IMAP

import os
from dotenv import load_dotenv
load_dotenv('.env')
username = os.getenv('USERNAME')
password = os.getenv('PASSWORD')

import imaplib

server = 'imap.gmail.com'
imap = imaplib.IMAP4_SSL(server)
imap.login(email, password)

print(imap.list()) # lists mailboxes
status, messages = imap.select(mailbox="INBOX",readonly=True)

Emails on the server are assinged sequential IDs. Starting from 1.

msg_uid = 10
status, msg = imap.fetch(str(msg_uid),'(RFC822)') # no clue why RFC822
# msg
#  0: msg itself
#  1: (uid, data)

import email
data = msg[1][1]
m = email.message_from_bytes(data)

print(m.get_payload())

Glossary

NOOP command (imap.noop()) is a test packet. Similar to a ping.

MIME is Multipurpose Internet Mail Extensions

['Delivered-To', 'Received', 'X-Received', 'ARC-Seal', 'ARC-Message-Signature', 'ARC-Authentication-Results', 'Return-Path', 'Received', 'Received-SPF', 'Authentication-Results', 'DKIM-Signature', 'X-Google-DKIM-Signature', 'X-Gm-Message-State', 'X-Google-Smtp-Source', 'MIME-Version', 'X-Received', 'Date', 'X-Account-Notification-Type', 'Feedback-ID', 'X-Notifications', 'X-Notifications-Bounce-Info', 'Message-ID', 'Subject', 'From', 'To', 'Content-Type']

Key	Description	Example
`Subject`	Subject line of the Email	"Shedule for Meeting"
`From`	Sender Email Address	sender-email@example.com
`To`	Recipient Email Address	recipient-email@example.com
`Date`	Date in Long Format with time zone	Sun, 19 Jan 2025 00:11:46 GMT
	More Obscure Fields
`Content-Type`	Media Type (Full List) boundary charset	`text/plain`
`Received`	by [IPv6] with [Protocol] id [alphanumberic]
`Delivered-To`	Recipient Email Address	recipient-email@example.com

Dealing with Multipart Emails

Content-Type: multipart/alternative, mutlipart/mixed, etc.

Multipart Emails contain multiple parts with different types. And many email providers/clients don't bother minimizing the type, just always use the universal multipart.

.is_multipart()

Iterator: Warning! first object is parent itself!

.walk() generator object to iteratre through all message parts.

"part" object:

.get_content_type() (get subtype)
.is_multipart() (can be nested)

Attachments

Actions

FLAGS:

Marking Read/Unread

Dealing with Secrets

Never ever hardcode snsitive information, especially when publishing it. Put whatever holds the secrets into .gitignore.

TOC:

dotenv (Python)

Dotenv

L main.py
L .env

PASSWORD=password1234

import os # necessary
from dotenv import load_dotenv

load_dotenv('.env') # Loads into OS

password = os.getenv('PASSWORD') # Gets from the OS

(os.getenv("HOME") is Bash environmental variables $ env)

YAML

Android Termux

Default package manager is pkg

But it doesn't work for 99% of stuff, so expect to compile and git clone stuff yourself. Also, superuser is broken unless you have rooted your phone.

SSH server

SSH = Secure Shell (encrypted connection to another computers commandline)

Official docs on Remote Access.

Doing everything from the computer is 10x easier, so use ssh.

In Termux

Install OpenSSH pkg install openssh
Host with sshd (opens on port 8022)
1. Warning, you are exposing your phone to the world here.
whoami should give current user (something like u0_a123)
Setup password with passwd (Remember step 1.1, so don't set defaults)
Do whatever you wanted to do.

Last. pkill sshd to close

Hosting stuff

Only real usecase of using a phone for selfhosting, is when you don't have your laptop, but other people for some reason do, and you want your phone to do the heavy lifting.

Figure out IP you will be hosting stuff from:

Localhost: 127.0.0.1 or localhost.
Local (home network): ifconfig
Public:

Awesome Selfhosted

What I managed to get running:

Atheos Web IDE

Github

Normal install won't work because Termux doesn't like sudo.

Atheos is written in PHP. So

Hence git clone https://github.com/Atheos/Atheos

cd Atheos

php -S [your local ip]:8000 (local network php server on port 8000)

Open [phone's local ip]:8000 in your computer browser and enjoy.

JupyterLab Python Notebook

Github

Given how Termux doesn't like pip, and how JupyterLab + Kernels are generally hard to get working, messing with JupyterLab will likely be futile.

Try pip3 install jupyterlab at pray that it works.

VS Code

Github

You can do VS Code online at vscode.dev, only caviat, you need to pay Github for hosting, or provide your own server.

At this point, consider signing up for Replit or some other online IDE. But if you are really set on hosting from your phone: they conviniently have a tutorial on their official website. (tldr pkg install code-server)

No JupyterLab because pip breaks
No Marimo because pip breaks

Python

Yes, pip constantly fails to install stuff

Compiling C Projects

Read the damn docs and compilation instructions if available.

Classic

Classic way of compiling almost any project written in C.

./configure && make && make install

Breakdown:

./configure executes configure file to check dependencies

! Note that configure often doesn't exist initially, and a autogen.sh (or similar) must be run (bash autogen.sh) to generate configure and other setup stup steps.

make

make install

Docs

Instructions for compiling/building/running are usually in README.md, INSTALLING.md, or a similar file. cat to read them.

The docs for small projects are usually outdated, files might have been moved around within the source code.

Use tree command to see what files there are.

Search for mentioned files in the source code via fuzzy finder fzf or find . -name "autogen*" (find command options starting/path Regex)

C Libraries

pkg-config --list-all to see all libraries you have installed.

If you need to install a library, you can compile it yourself using pretty much the same steps here.

LittleJS

Lynx Browser

Famously cool browser, confusingly difficult to run.

Official Lynx Website has the latest stable version listed, as of writing it is 2.9.2.

Installing

Package Manager

Lynx is fairly popular, so most package managers have a lynx package. (see here)

You can check the version with lynx --version

Compiling it yourself

w3m

w3m is a CLI web browser written by a Japanese guy.

It is hosted on SouceForge.

Like most other C project, run ./configure && make && make install in the soure code directory.

Elinks

https://github.com/rkd77/elinks download latest release from releases (as of writing v0.18.0)

./configure && make && make install

you might notice ./configure doesn't work

Run ./autogen.sh and it configure will appear.

.configure should produce a list of features that will and won't work based on prerequisite libraries. (Even if some are missing, that should be ok.)

make should compile and list bunch of linking (LD and CC) for the above libraries.

make install should put the final binary into your bin folder.

elinks should work now

Installing Libraries

"tre" Regex Library

Browsh

Compiling C Projects

Read the damn docs and compilation instructions if available.

Classic

Classic way of compiling almost any project written in C.

./configure && make && make install

Breakdown:

./configure executes configure file to check dependencies

! Note that configure often doesn't exist initially, and a autogen.sh (or similar) must be run (bash autogen.sh) to generate configure and other setup stup steps.

make

make install

Docs

Instructions for compiling/building/running are usually in README.md, INSTALLING.md, or a similar file. cat to read them.

The docs for small projects are usually outdated, files might have been moved around within the source code.

Use tree command to see what files there are.

Search for mentioned files in the source code via fuzzy finder fzf or find . -name "autogen*" (find command options starting/path Regex)

C Libraries

pkg-config --list-all to see all libraries you have installed.

If you need to install a library, you can compile it yourself using pretty much the same steps here.

Tre Regex Library

https://github.com/laurikari/tre

git clone https://github.com/laurikari/tre

./utils/autogen.sh (creates configure executeable among other things)

./configure
make
make check
make install

Compiling C Projects

Read the damn docs and compilation instructions if available.

Classic

Classic way of compiling almost any project written in C.

./configure && make && make install

Breakdown:

./configure executes configure file to check dependencies

! Note that configure often doesn't exist initially, and a autogen.sh (or similar) must be run (bash autogen.sh) to generate configure and other setup stup steps.

make

make install

Docs

Instructions for compiling/building/running are usually in README.md, INSTALLING.md, or a similar file. cat to read them.

The docs for small projects are usually outdated, files might have been moved around within the source code.

Use tree command to see what files there are.

Search for mentioned files in the source code via fuzzy finder fzf or find . -name "autogen*" (find command options starting/path Regex)

C Libraries

pkg-config --list-all to see all libraries you have installed.

If you need to install a library, you can compile it yourself using pretty much the same steps here.