Gene Cooperman's
Computer Science Information
Think of these pages as my current set of bookmarks, except that I've
made them public, in case somebody else finds them useful.
- WaybackMachine.org
(see sites as they previously were)
- arXiv.org e-Print archive
- CiteSeer (indexes papers and citations)
- Northeastern
University Libraries
-
DBLP Bibliography
(alt
or alt2)
- Cloud Mining - DBLP
(searches DBLP database, w/ enhancements from CiteSeerX)
-
crossref.org: free DOI guest search
- IEEE Xplore
-
N.U. Research Databases (via N.U.; incl.
IEEE
Electronic Library,
JSTOR,
SpringerLINK,
MathSciNet, AMS, SIAM, ACM Digital Library, etc.)
- ACM Portal
- Free online ACM courses
-
CiteSeer (Computer Science Papers, ResearchIndex)
- Google Scholar
- Pastebin.com (anonymously create
temporary web pages, and even sub-domains, with pasted text inside)
- Dropbox (transfer files using storage there temporarily)
-
The Quick Reference Site (python, gdb, doxygen, php, Java,
SQL, bash, C++, one-page Linux reference, ...)
-
Welcome directory (FAQ's)
- On-line exercises
- Project Euler
(Programming puzzles for combinatorics, etc.)
- smashthestack.org
(interactive tutorial on how to do exploits)
- Exploit Exercises
- CSAW CTF (Capture the Flag)
- CodingBat Python
- Literate Programs
(variety of solutions to standard problems in multiple languages)
- Project Jupyter
(enables interactive code w/ graphics, video, LaTeX, etc.,
to run in web browser. Supports Python, R, Julia, Scala,
and many others; also supports notebooks, handles LaTeX notation,
etc.)
-
High Performance Software (HiPerSoft)
-
Analysis of SpamThru bot controller
-
Anti-spam message headers in Microsoft 365
-
How Microsoft 365 uses Sender Policy Framework (SPF)
to prevent spoofing
- Microsoft Outlook / Office365 URL Rewriting in E-Mails
-
Publications of Joe Stewart (spam analysis)
- Spamcop.net
- Anti-SPAM group
- Disposable SMS services
-
Computing Languages (UK)
- Go Programming Language (aka golang)
(clean version of C; principled, safe use language;
garbage collection; keeps syntax of pointers, with python-style slices
and STL-style maps (hash arrays);
anonymous functions
(
go func(): "lambda functions" including closures),
concurrency based on CSP channels; methods (type signatures)
without classes (the method set of a type can be viewed as a class),
and use of interfaces for internally derived type hierarchies
instead of explicit object oriented inheritance;
gcc back end, yacc/bison parser)
- Plan 9
operating system from Bell Labs from 1990s
(
Plan 9 compiler suite)
- Haskell
(pure functional, strongly typed language with lazy evaluation
--- reminiscent of PROLOG and pure Scheme,
with a tutorial, A Gentle
Introduction to Haskell; and extensive Wiki, e.g.
Haskell Books)
- Poly/ML (implementation of SML,
including a
debugger)
-
Common Lisp the Language, 2nd Edition
-
Common Lisp the Language, 2nd Edition (alt)
-
ANSI Common LISP standard: discussion record
-
General LISP archive (incl. CLISP, Cltl2, dpAnsii, etc.)
- GCL
(GNU Common LISP)
-
ECL (Embeddable Common Lisp)
(or alt)
- EcoLISP
(alt)
(with thread support, KCL derivative)
- CLISP
(or alt)
-
CLiCC: The Common Lisp to C Compiler
(standalone C applications, no EVAL)
-
Some LISP tools
- PERL home page
(also see PERL tutorials/references)
- Transformer Links
-
Pratt parser in Python
li>
How to parse a json file from Linux command line using jq
-
The C++ Virtual Library
-
C++ Annotations (Excellent!! For knowledgeable users of C core,
basics, pointers, structs, arrays, etc.)
-
C++ "placement new" examples
- Bit arrays in C (not C++, but useful)
- Debugging C++ with GDB:
-
Object-Oriented Programming With ANSI-C (by Axel-Tobias Schreiner,
1993; on-line book) (Excellent for understanding low-level
implementations)
- gcc Wiki - visibility
(
__attribute__ ((visibility("hidden"))) and
g++ -fvisibility=hidden, but C++ ISO standard
says new and delete must
always have default visibility)
- cppreference.com
(for brief summary of libraries, STL, I/O, string class)
-
comp.lang.c Frequently Asked Questions
-
Annotations for ANSI C Standard
-
Programming in C (high quality pointers,
including ANSI C grammar for
Yacc
and Lex)
-
The C Book (based on ANSI C 91 standard)
-
Draft standard for C11> (April, 2011)
-
C++ Reference (C, IOStream, Strings, STL, misc. libraries;
nicely organized!)
-
Dinkum C/C++ Library Reference
-
Sergey Bratus's C++ pointers
-
C++ Public Review Document (1997, html)
-
C++ Draft Standard (November 1996 Working Paper)
-
C++ Standard ANSI Draft/ISO Working Papers
-
Using and Porting GNU CC
-
Intel C++ compiler for Linux (free for non-commercial)
(features
such as profile-guided optimization)
-
Pointers to STL sites (Standard Template Library for C++)
- Boost (library of useful routines
for C++, including serialization)
- Documentation
of Java classes and methods
- JDance
("online Java information center", pointers to info on the Web)
- Java books (some may be free via Safari subscription)
-
Java API
for all Java classes (Standard Edition v. 1.4.2)
(Sun Microsystems, Java 2 Patform, Standard Edition v. 1.4.2)
-
Java example programs to download (java.io, java.lang, java.util,
etc.)
-
Java Precisely (with old edition available for free download;
essential points of Java, with matching concept and example,
in only about 100 pages)
-
Effective Java (by Joshua Bloch, good for advanced Java,
with sample chapters on Substitutes for C Constructs, and
Methods Common to All Objects)
- Java Tutorials
by Sun Microsystems
-
Fortran 77 Tutorial
-
Fortran tutorial
-
MPI FORTRAN90 Examples
-
MPI by examples in Fortran
-
Problems With Fortran Bindings for MPI
(MPI 2.2 report (standard))
- Z notation for
formal verification
- SAT solvers
- Explicit State Model Checking
-
Separation Logic
- BDD (Binary Decision Diagram)
-
Formal (Finalized) Specifications of OMG (Object Management
Group), including many Corba standards
-
OOP Design Page
- Unified
Modeling Language Resource Center
-
Booch Method Overview
Parallel Stuff
- Parallel research projects (that are or were interesting to me)
- Cluster and Wide-Area Computing
- PlanetLab
(sharing of CPUs; an alternative to the Grid;
donate two computers and get free access to huge network for
parallel computation)
- Condor
(submission of remote jobs, automatic nice, migration, etc.; including
downloads and
manual)
- The Globus Project
(technologies for computational grid)
- Beowulf.org
- Beowulf Underground
- MOSIX
(transparently load balances: application runs on available node
of cluster)
- Rocks Cluster Distribution
(with
introduction;
automatically set up Redhat Linux on cluster using Redhat Kickstart)
- Newer experimental low-power computers for high performance clusters
-
The AppleTV Cluster (cluster of iPads)
- Raspberry Pi
(ARM-based Linux computer, 512 MB, about 1 Watt consumed,
about $25, 2011)
- Calxeda (5 Watts/server with
4-core system on a chip; 4 GB dedicated DDR3 DRAM;
32 KB L1 cache, and 4 MB L2 cache;
4 servers
on mini-board w/ 10 GBEthernet interconnect; Supports
H-P Redstone w/ 6 mini-boards
plug into a 2Ux0.5U tray (4 trays fit in 4U)
-
Legion Home Page (World-wide virtual computer)
(with its own startup company:
Applied Metacomputing LLC)
-
Benchmarks:
- Parallel Languages
- OpenACC
no-ip.com (free subdomains redirecting to
dynamic DNS, and other (for fee) services)
Google Analytics: login and check map, if registered
The BatLab (Build and Test Lab, formerly NMI)
openSUSE Build Service
(submit an openSUSE package, and it gets converted, built and tested
for many Linux distributions)
Roomy sourceforge
DMTCP sourceforge
URDB sourceforge
CRIU
Checkpointing Ideas and Systems
-
Los Alamos National Laboratory Computer Science Reearch HPC-5
(Fault tolerance and system resilience)
- DMTCP (Distributed MultiThreaded
CheckPointing) (entirely in user space, supports
TCP sockets, Matlab, distributed computations, MPI, etc.)
-
Berkeley Lab Checkpoint/Restart (BLCR) (based on Linux kernel
module, supports MPI through callbacks from MPI implementation,
integrates with several batch queue systems)
- Jaryba SmartSuspend
(Suspends a live process, giving back memory to O/S
and releasing software license), but apparently doesn't checkpoint)
-
XEmacs: A Portable Unexec Replacement (For
checkpointing, emacs unexec.c provides a classical
system-dependent solution, but an alternative is to
create one or more large initial static buffers plus
a redefined malloc that uses memory in the static buffer;
and then do all memory allocation into the static buffer.
On dump, write to file, on revive, copy to original absolute address.)
-
Checkpointing Research at Tennessee (Jim Plank et al.)
Scalable Coherent Interface (SCI)
(allowing fast remote memory access; largely superseded by
InfiniBand, Myrinet, and iWARP)
Tutorials on DSM and
other topics (by Veljko Milutinovic)
Bill McColl Home Page (Publications on making
parallel programming more practical, BSP, chair of
Oxford Parallel group (industry-academia))
patterns (Parallel Computing Laboratory, Berkeley,
with Our Pattern Language (OPL)
(for parallel patterns, from Berkeley and UIUC)
BRICS
(Semantics of programming and parallelism)
Lecture Notes on "Advanced Programming Languages"
(Andrew Myers, 2009)
Low-level Systems, Debugging, and Instrumentation tools
- Valgrind
(for finding memory leaks, etc., but also for profiling:
--lead-check=yes --num-callers=8 --gdb-attach=yes
--show-reachable=yes )
-
ThreadSanitizer (data-race test; and available in gcc-4.8 and
above via -fsanitize=thread)
-
AddressSanitizer: a fast memory error detector
(compiles code for shadowing instead of valgrind interpreted approach)
-
Standalone checkpointing (from Condor)
- Paradyn Parallel Tools Project
(U. Wisc., many binary-based tools)
-
Injectso 0.2.1 (shared library injector; search for link on web page)
(or alt)
- dynInst, from U. Wisc./U. Md. (supports code injection;
assumes presence of libdl.so in application)
- ksplice
(live patches to a running kernel)
- PIN,
tool for dynamic instrumentation of executables
(or alt)
- DynamoRio (dynamic Instrumentation
tool, runtime code manipulation, open source)
- Frida (interactive function
tracing (wrappers) for API of application, including network calls)
-
Debugging Linux Applications (Nice overview of
gdb/objdump/nm/strace, etc., etc.; requires facebook account
to download)
-
Troubleshooting apps in Linux (from Red Hat)
-
Defensive Programming (by Ulrich Drepper)
-
Dtrace for Linux
-
Sytemtap (probes internal kernel actions in a running Linux;
investigating probing userspace applications)
- LTTng (Linux Trace Toolkit next
generation)
SAL - Parallel Computing Programming Languages and Systems
ACTS Tools (ScaLAPACK, PETSc, etc.)
Freely Available Software for linear Algebra on the Web
(EXCELLENT: LAPACK, direct/iterative, preconditioners,
sparse packages, BLAS, distributed solvers, etc.)
Supercomputing and Parallel Computing Resources (CMU and other)
Collectl (collect performance data for
cpu, disk, memory, network, etc.)
The Berkeley NOW project
San Diego Supercomputer Center (SDSC)
National Computer Science Alliance (NCSA), Urbana-Champaign
NSF Center, NPACI
(National Partnership for Advanced Computational Infrastructure:
archives)
(supercomputing center in France of the Ministry of Higher Education
and Research)
OpenFabrics Alliance
(infiniband et al.; including OFED software:
Open Fabric Enterprise Distribution)
GloMoSim (network simulator based on Parsec, which is a
parallel event simulator)
IBM C++ Compiler
X11 (X Windows)
- X Window System protocols and architecture (Wikipedia)
- Xming implementation
of X-Windows for Microsoft Windows
- Xlib (libX11.so on UNIX)
(Wikipedia,
windows and subwindows; buttons, menus, scrollbar, etc., handled
by libraries on top of Xlib)
- XCB (lower level
substitute for Xlib)
- X Window System core protocol
- Session mgmt:
xsm (X11 demo session manager), gnome-session, etc.
(Apps attach to a session, and then respond to checkpoint
and restore requests).
- X11 virtualization
- Security, authentication, miscellaneous tools, and other software
SCone User Guide 4.1.0 (examples of doing small, simple builds;
but go directly to the appendix for the full documentation)
Bazel - a fast, scalable, multi-language and extensible build
lldb - gdb for Mac, comes from LLVM (e.g., for clang compiler)
Cygwin FAQ
OpenVAS (vulnerability
scanner and manager; open source)
SELinux tutorial
JPlag
(with
downloads here)
MOSS
(plagiarism detection)
Interesting hardware
Supercomputing Resources
DOE Exascale applications for HPC
SCV at B.U. (SCV = Scientific Computing and Visualization;
account requests and
administration;
facilities part of Mariner
project -- New England metacenter)
Linux Community Development System (IBM S/390 mainframe, 16 CPU's)
National Coordination Office for HPCC
PRACE (Partnership for Advanced Computing in Europe)
(HPC supercomputing consortium in Europe)
Earth Simulator
(NEC Supercomputer)
Designing and Building Parallel Programs (based on book
from 1995)
Bioinformatics
Data-Intensive Computing
- FAWN
(Fast Array of Wimpy Nodes; fast, scalable, power-efficient)
-
Global Arrays (with emphasis on quantum chemistry,
linear algebra, MPI)
Symbolic Algebra Packages
- SAGE (Software for
Algebra and Geometry Experimentation: glues together
GAP,
(Groups, Algebra and Programming),
Maxima
(general symbolic algebra, like MAPLE and Mathematica),
SageMath (with Sage notebooks),
GMP (GNU Multiple Precision
arithmetic),
MPFR
(mult. prec. floating point, support for interval arithmetic),
The CS 6120 Course Blog: The Cult of Posits
(by Dietrich Geisler, Edwin Peguero December 3, 2019)
PARI
(algebraic number theory),
NTL (Number Theory
Library),
Singular
(Gröbner bases and algebraic geometry),
Gnuplot
(for graphings, see also,
SLxfig (excellent,
part of S-Lang),
R, Octave and Jgraph (search for Jgraph on this page)),
Octave,
Numerical Python (for efficient numerical analysis),
etc.,
and some non-free packages (
Magma,
MAPLE,
Mathematica),
via Python and some glue code)
- GAP (U.K.)
(Groups, Algorithms, and Programming)
-
Overview of MAGMA
-
MAGMA manual
- Eamonn O'Brien's
Algebra Database in BibTeX
-
Rob Wilson's atlas of finite group representations
-
John Bray's list of group presentations
- NTL (finite field arithmetic)
- Maxima
- MuPAD (symbolic algebra system)
- Yacas,
Yet Another Computer Algebra System (RULE-BASED SYSTEM:
free system, real and
complex polynomials and elementary functions, symbolic limits,
derivatives and (limited) integration, symbolic solution of (simple)
equations, and some special mathematical functions)
- Macaulay 2
- ACL2
(A Computational Logic for Applicative Common Lisp)
- HOL4 (Higher Order Logic 4)
Researchers/groups/people with some interesting home pages
(not an exhaustive list :-) )
-
Yale FLINT Group (Zhong Shao, lead; certified systems software)
- Related Projects
(Verisoft, CompCert, Boyer, Coq, Isabelle/HOL, ML, Nuprl, SPIN, ...)
- Saïd Tazi
- ROMA (ENS-Lyon/INRIA/CNRS)
- Frédéric Vivien
- Franck Cappello
(or alt)
- lil Drira
- Yuanyuan Zhou
- Shan Lu
- Derek Holt
- Mark Jerrum
-
Bill Kantor
- A. Kerber and R. Laue
- Eamonn O'Brien
- I. Pak
- Klaus Schauser
-
Elias Manolakos (Par. Proc. and Arch. Res. Group)
-
The Probability Web
- God's Number is 20
- George Havas
-
Coset Enumeration
-
Markus Püschel (computational representation theory)
-
Egner (computational representation theory)
-
Chi squared table of critical values
(and alt;
and
other tables)
-
Mathematical Atlas: A gateway to Mathematics
(and
alt)
- MathWorld
(math encyclopedia, varied depth)
-
Math Websites (excellent collection from Art of Problem Sovling)
- Sergey Bratus's
Home Page
(to be continued)
Online C.S./technical books
-
Implementation of mutex using futexes
- Open Textbook Library
-
Machine Learning Open Book
(Excellent overview of Machine Learning/Deep Learning)
-
The C Book (by Mike Banahan, Declan Brady and Mark Doran;
excellent!)
-
Operating Systems: Three Easy Pieces
(or alt),
-
Redis in Action (in-memory database)
-
Linux Inside (a github book by "OxAX")
-
(Excellent!:
Shichao An's Notes on Linux Kernel Develpment book (by Robert Love);
and see notes on networking, Go Language, etc., at same site;
ALSO SEE: Github notes on a still wider variety of subjects)
-
xv6, a Simple, Unix-like Teaching Operating System (pdf)
-
xv6 (home page and pointer to the software)
-
xv6, rev7 (pdf: booklet with line numbers for source code)
- 6.828: Operating
System Engineering (MIT, uses xv6)
-
Beej's Guide to Network Programming: Using Internet Sockets
- The Art of HPC
(by Victor Eijkhout of TACC; 4 volumes; excellent)
-
Parallel Programming for Science Engineering
(by Victor Eijkhout)
-
Is Parallel Programming Hard, And, If So, What Can You DoAbout It?
-
Specifying Systems: The TLA+ Language and Tools for
Hardware and Software Engineers (by Leslie Lamport)
-
SSA-based Compiler Design (pdf: Static Single Assignment Book:
compiling dynamically typed languages into static types)
- Great Theoretical Ideas in Computer Science (CMU: CS 15-251)
-
Great Ideas in Theoretical Computer Science (MIT OpenCourseware: 6.080/6.089)
-
Network Science Book (From babasilab.neu.edu)
-
Linux and open source tips (quickly browse to pick up useful
tips you might have missed)
-
Safari Tech Books Online
or alt (Requires Northeastern U. account;
includes O'Reilly, Novell Press,
Addison Wesley, Prentice Hall, Sun Press, SAMS, Microsoft Press,
Macromedia Press, Adobe Press, etc.)
- Operating Systems
- W. Richard Stevens' Home Page
(including his technical books freely available online)
- The XINU Page
(textbook and software - 8000 lines of code)
-
Operating Systems: Three Easy Pieces,
Arpaci-Dusseau and Arpaci-Dusseau (free on-line,
and cheap softcover available, from U. Wisconsin)
- Computer Systems: A Programmer's Perspective,
Bryant and O'Hallaron (from CMU)
-
Introduction to Computing Systems: From Bits and Gates to C and
Beyond, 2/e (© 2003, with simulator)
- Operating Systems Concepts, Silberschatz, Galvin and Gagne
-
The Datacenter as a Computer: An Introduction to the Design of
Warehouse-Scale Machines, Second edition Synthesis Lectures on
Computer Architecture
- Fundamentals of Parallel Computer Architecture (slides)
- C in a Nutshell, Prinz and Crawford
-
Digital Library of Science (free tech. books in math and sciences)
-
List of Free Online Books, Tutorials and Lecture Notes
- Open Textbook Library
(excellent, but should also refer to ostep.org}
-
Links to free math books (from PlanetMath)
-
Other Free books (besides GNU manuals)
-
GNU gcc readings on systems info
-
Mythical Man Month (by Fred Brooks)
-
SICP, Structure and Interpretation of Programs
-
Principles of Computerized Tomographic Imaging
-
Elementary Calculus: An Approach Using Infinitesimals
by Jerome Keisler (rigorous infinitesimals instead of epsilon-delta)
-
Introduction to Modern Cryptography
-
Beej's Guide to Network Programming: Using Internet Sockets,
by Brian "Beej" Hall
- Networking: O'Reilly's "Linux Network Administrator Guide" or
"Essential System Administration" or the HOWTOs and guides
at www.tldp.org
- Using CSP
Hoare's 1985 edition of the book on Communicating Sequential Processes
(and several chapters from his most recent book,
The Theory and Practice of Concurrency)
- 1024 cores: All about lock-free concurrency, etc. (unrelated Intel Blog)
- Mostly Lock-free Malloc (Dice and Garthwaite)
- DWCAS in C++
(excellent!)
- Fear and Loathing in Lock-Free Programming
(by Tyler Neely, Oct., 2017)
-
CppCon 2014: Herb Sutter "Lock-Free Programming (or, Juggling Razor Blades), Part I"
(pdf)
-
Synchronization Without Locks
(Charles Leiserson, Fall, 2018, MIT 6.172, Perf. Engineering)
-
C11 Lock-free Stack (Sept., 2014; Excellent! Discussion
of dwcas, lock-free stacks, hazard pointers, etc.))
-
ARM LL/SC exclusive access by register width or cache line width?
(stackoverflow)
-
128-bit Atomics Are Practical Now (with lock-free support)
- Writing Concurrent Systems (basics, but well-written)
-
Bug 110061 - libatomic: 128-bit atomics should be lock-free on AArch64
(still unresolved as of May, 2024)
- Lock-free algorithms (first in a series of blogs, all worth looking at)
- For sockets, see also
and click on for UNIX socket FAQ
or other
or
Debugging TCP (excellent!)
-
A Deep Dive into the SIGTTIN / SIGTTOU Terminal Access Control
Mechanism in Linux
-
memory map file with growing size
(StackOverflow, but nice overview/testing for using PROT_NONE
to mmap extensions of a file, and then grow the file later;
libsodium-23.3 on ARM64, Debian-12, Liux-6.6 seems to play tricks
like this, although it's owned by root; libsodium-23.3 on Ubuntu-23.04
for x86_64 doesn't do this.)
-
How TCP backlog works in Linux
-
Casual overview of SO_KEEPALIVE
-
Preserving disconnection due to network inactivity (and
"Using TCP keepalive under Linux")
(part of "TCP keepalive HOWTO" from tldp.org)
-
TCP Keepalive Best Practices - detecting network drops and
preventing idle socket timeout
-
A compendium of NP optimization problems
(from book, Complexity and Approximation)
-
Introduction to the New Mainframe: z/OS Basics
-
The C Book by Mike Banahan, Declan Brady and Mark Doran
(ANSI C (1991), similar to Kernighan and Ritchie book,
but more casual style)
-
C++ Annotations, by Frank B. Brokken
(Excellent!! For knowledgeable users of C core,
basics, pointers, structs, arrays, etc.)
-
The Art of UNIX Programming (by Eric Steven Raymond),
excellent overview of style and philosophy in UNIX
-
Linux System Programming (by Robert Love, 2007;
on-line through Northeastern subscription to
Safari Books Online)
-
Thinking in Java/C++/Patterns/Enterprise Java
by Bruce Eckel
(excellent tutorial-style books for intermediate programmers;
Note also Thinking in C# by Larry O'Brien
and Bruce Eckel, Prentice Hall 2003, hardcopy only)
(Also available here)
-
Java Precisely, by Peter Sestoft (with
full book)
-
On Lisp. Advanced Techniques for Common Lisp, by Paul Graham
- Linkers and Loaders
by John Levine (free version,,and
for-sale copy)
-
Linker and Loader Libraries Guide
-
Lecture 05: Linking and Loading (excellent)
-
Linux inside (linker chapter)
- From Zero to main(): Demystifying Firmware Linker Scripts
(excellent)
- The most thoroughly commented linker script (probably)
-
Actual default linker script and settings gcc uses"
-
default-linker-script.txt (using "ld --verbose")
-
Using ld: The GNU linker
(ld --verbose for default linker script;
ld -T LINKER_SCRIPT for replacement linker script)
- Runtime Linker (Oracle)
- Runtime library search:
-
fully_qualified_path_with_slash->RPATH->LD_LIBRARY_PATH->RUNPATH->paths_of_link_editor_with-L->/etc/ld.so.cache->/lib->/usr/lib (or maybe lib64 for lib)
NOTE: -L options are prepended to the default path,
lib:/usr/lib, and in the order specified by the
link editor ('ld'). All '-L' optiona are
applied before any of the '-l' options.
NOTE: LD_LIBRARY_PATH can also affect the behavior of ld
in combination with -L. See man ld for details.
NOTE: ldd shows you the final link, and it takes account of
LD_LIBRARY_PATH
Linux Online Books
-
Loads of Linux Links
-
LINUX: Rute User's Tutorial and Exposition
(and alt;
excellent for Linux or UNIX!!!
Broad survey of everything you wanted to know
but were too shy to ask.)
- Controlling terminal
-
Overview of Linux Boot Process, etc. (excellent!)
- IBM Technical Information
- LinkinWiki (Linux kernel, etc.; e.g., see slides for Linux Kernel 3 for
"Understanding the Linux Kernel" book)
- The Linux kernel
-
Linux Kernel links
- Linux kernel books (online and hardcopy)
- The Linux Kernel 2.6 (excellent!, by Andries Brouwer, current to Linux 2.6.9; overview of libc interface, syscalls, signals, ptrace, /proc, vfs, memory overcommit, async events, etc.)
-
Linux man-pages home (documents Linux kernel in man pages)
-
/proc filesystem documentation
- Overview of ramfs and tmpfs filesystems
-
alternate directory of Linux books/articles
- Kernel Trap
(articles about the Linux kernel and related stuff)
-
The Linux Kernel (based on 2.0)
-
Linux Kernel 2.4 Internals
-
The Linux Kernel Hackers' Guide (based on 1.0)
-
Tour of the Linux kernel source (based on 1.0)
-
Conceptual Architecture of the Linux Kernel
and
Concrete Architecture of the Linux Kernel
and
Linux as a Case Study: Its Extracted Software Architecture
(combination of last two as ICSE-99 paper
- kernel.org
(documentation on Linux kernel; excellent!)
-
hugetlbpage.txt (huge pages)
-
Tutorial on kprobe and jprobe for debugging Linux kernel
- Understanding the Linux Virtual Memory Manager (748 page book; 200 pages
of text and the rest code)
(or: here)
- Hack the Virtual Memory: malloc, the heap & the program break
-
Understanding Virtual Memory
-
Overview of the Virtual File System
-
Linux Device Drivers
-
Implementing virtual system calls (vDSO and vvar memory sections) (Linux Weekly News)
- Also, search for TLS, TCB, for issues of multiple threads
-
"The Definitive Guide to Linux System Calls"
- x86-64 order of passing parameters in registers
- Kernel Traffic
(weekly news reports of latest developments for Linux kernel,
for Wine, and for ReactOS);
see also Kernel Trap
-
LinuxChanges - KernelNewbies Wiki
- KernelTrap.org
(News and Forums about Linux kernel)
-
Handbook of Applied Cryptography,
by Alfred J. Menezes, Paul C. van Oorschot and Scott A. Vanstone
-
Data Mining Survivor: Data Mining Dekstop Survival Guide
(see also
Data Mining Catalogue)
- Graph Theory, by Reinhard Diestel
- Abstract Algebra
by Beachy and Blair
-
Templates for the Solution of Linear Systems, 2nd Edition
(Numerical Analysis)
-
Iterative methods for sparse linear systems;
and Numerical Methods for Large Eigenvalue Problems,
both by Yousef Saad
- Computing with Floating Point
(by Florent de Dinechin, as slides,
but excellent overview of the issues)
-
Permutation Group notes and preprints (excellent;
part of more diffuse
Permutation groups resources, all maintained by Peter Cameron)
-
Art of Assembly Language Programming (Intel assembly)
-
DevCentral Learning Center (tutorials)
-
International Legal Protection for Software
-
The Elements of Statistical Learning: Data Mining,
Inference, and Prediction
-
Engineering Statistics Handbook (NIST, free and online)
- Resampling method (statistics)
-
Designing and Building Parallel Programs (Online)
-
Implementing Mathematics with the Nuprl Proof Development System
-
Linear Algebra Applications
- Also, see Common LISP books
-
O'Reilly Nutshell books
-
Linux Tutorial: POSIX Threads
(see also
NPTL-Design.pdf)
-
Bil Lewis's course slides on threads (excellent)
(see also his
Omniscient Debugger, A Java debugger that can move backwards
through debugged program)
- Jockey
(free software for execution record/replay; works by
trampolines after parsing binary programs as assembly;
pdf)
-
SmartGDB (not under current active development?)
-
Simultaneous Multithreading (SMT) Home Page
(a combination of ILP and TLP -- when insufficient ILP,
other threads are executed)
-
HyperThreading (Intel) (Intel implementation of SMT
(Simultaneous MultiThreading),
for most recent Intel processors)
- Microsoft Windows
- NOTE: In Windows 11, if a simple graphics command
(e.g., gedit, xeyes) hangs and does not display, there are
several possible fixes to try (below). (I do not use the
special Intel/AMD driver for GPU-accelerated graphics.)
-
wslg: Can't open display: :0 + X11 server is not running
(Microsoft github issue with many user comments)
- Run wsl.exe (or put it in your ~/.bashrc).
(Possibly, this is not needed if you uninstall and re-install
your Ubuntu or other Linux distro.)
- Is your DISPLAY environment variable set to :0?
- Maybe, does ls -l /tmp/.X11-unix point to
/mnt/wslg/.X11-unix? (But mine can also work without that.)
- Does /mnt/wslg/.X11-unix/X0 work? (Is the Xserver running?)
- If weston-subsurfaces or weston-flower working? (Possibly,
the problem is with the OpenGL library.)
- If graphics works, but not sound, see the link to get
PulseAudio working (below).
- If trying to use graphics/sound by using the older
xlaunch-genie.sh for Windows 10, then make sure to first
run xlaunch-genie.sh
-
How do I get back unused disk space from Ubuntu on WSL2?
(and wslcompact utility)
- How To Get Sound (PulseAudio) To Work On WSL2
(March 8, 2021, from Linux Uprising)
(or alt from CodePre.com)
(or alt from Linux tips Blog, Sept. 3, 2021)
-
How to Run Linux GUI Applications on Windows 11 Using WSL
-
-
Windows Subsystem for Linux Installation Guide (WSL2)
-
WSL2: Launch command & configuration
-
Mixing Linux and Windows commands
- Welcome to WSLg
(from Microsoft)
- How to Install Ubuntu Desktop With GUI and Sound on WSL2 (This installs sound; but shouldn't WSL2 (Windows build 20000) support sound automatically?)
-
WSLg (for Linux GUI) (of Windows 21362 build; obsolete?)
-
Installing a Linux GUI (desktop) for WSL2
-
Fedora Linux for WSL2 (MS Store currently has Ubuntu, SUSE, but not Fedora; Nov., 2020;
For /etc/resolv.conf, I used the Google public nameservers; 'dnf' is the Fedora package manager.)
-
Updating the WSL 2 Linux kernel automatically
-
Docker in Windows Subsystem for Linux (older article)
-
Easiest way to run GUI apps on WSL as of 2018 (older article)
Sharp Keys: Many Windows laptops seem
to place the up-arrow key exactly where the right-shift
key should be. This makes it almost impossible to
touch-type. To fix this, download Sharp Keys.
See for an explanation about the Windows regostry..
Ways to Install Windows 11
(ways to install even if not compatible; at your own risk;
if choosing "full install", possibly an option to revert to
Windows 10 within 10 days)
Blogs
On-line C.S./technical courses
C.S. Outreach
On-line C.S. course syllabi
-
The Tab Completion Grade Book (Sourceforge,
Cross Platform based on Java)
-
OpenGrade, Cross Platform (based on Perl, for
Linux and Windows but Windows version not officially supported)
-
Ggradebook (GNU Gradebook, requires GTK-1.2 (sic) or later
and dates on files are from 2000)
Useful off-line books (not an exhaustive list :-) )
- David R. Butenhof. Programming with POSIX threads. Addison Wesley,
1997
-
Java Precisely (with old edition available for free download;
essential points of Java, with matching concept and example,
in only about 100 pages)
-
Effective Java (by Joshua Bloch, good for advanced Java,
with sample chapters on Substitutes for C Constructs, and
Methods Common to All Objects)
- Effective C++, Addison-Wesley Profesional Computing Series, Scott
Meyers, 1998 (Second edition)
- More Effective C++, Addison-Wesley Profesional Computing Series, Scott
Meyers, 1996
- Design Patterns: Elements of Reusable Object-Oriented Software,
Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides,
Addison-Wesley, 1995
- Refactoring: Improving the Design of Existing Code
by Martin Fowler, Kent Beck (Contributor), John Brant (Contributor),
William Opdyke, Don Roberts
(with
online catalog of refactorings)
- Algorithmic
Number Theory
-
Randomized Methods in Computation
- G.H. Golub and C.F. Van Loan, Matrix Computations,
third edition, Johns Hopkins University Press, 1996.
(to be continued)
Ordering technical books on-line
Academic Journals
Miscellaneous
Solaris
GNU (GNU home page
with search)
- GNU ftp site
- GNU software (including some with no html manual, some that are
GPL, but not GNU)
-
GNU gcc readings on systems info
-
GNU software download (or
non-GNU software download;
or
mirrors at bottom of page,/a>)
- GNU manuals
- Some of these links are to specific version numbers. Check above
for latest version.
- Free Software Directory
of Free Software Foundation
-
GNU gcc C compiler
(see also gcj,
gcc for Java, and especially
The GCJ FAQ)
-
Using and Porting GNU Fortran (w/ gcc-2.95)
-
GNU C Preprocessor (cpp)
(also try "gcc -E")
-
The GNU C library (glibc, libc.a) (with
downloads; choose glibc-ports for ARM;
also good info for some C/UNIX
internals: malloc/free, obstack, descriptors/streams, sockets, device
files, tty internals, search/sort/pattern match, date/time,
signals, setjmp/longjmp, NSS(Name Service Switch: UNIX databases
such as DNS), assert, variadic functions; To compile, create
a build directory, and from the build directory, do for example:
../glibc-2.XX/configure CFLAGS="-g -O1" --prefix=$PWD ,
followed by make -j)
- eglibc svn repo
(Note that eglibc has been used instead of glibc by many distros.
But after eglibc-2.19, the plan is to switch back to glibc, with
some code from eglibc ported back to glibc.)
-
glibc 2.40 - doxygen documentation (Excellent!; if stale, refresh
link for latest glibc; Click on "Go to source code of this file")
-
glibc source by Woboq (Excellent!)
- Note that Debian and some other distros were using eglibc
(see eglibc-source package, and lzma
command (package xz-utils)).
- glibc
home page and wiki (including IRC channel, snapshots,
libc-help mailing list, etc.; To compile on UBUNTU, the following
is recommended:
CFLAGS=-O2 -U_FORTIFY_SOURCE -march=i486
-mtune=native -fno-stack-protector
)
- musl libc
(tries to be lightweight, fast, simple, and standards-conformant)
- diet libc
(a libc optimized for small size)
- bionic libc source
(Bionic libc is Google libc, based on BSD libc)
-
The GNU C++ Library (libstdc++) (including Parallel Mode and Multi-threaded allocator and debug mode)
- Static analysis
)
- (ii) Shared repository model (similar to svn):
git clone git@github.com:fred-dbg/fred.git fred; locally
modify; git commit; git pull; git push origin master
Travis CI build pipeline
SVN
- On-line svn book
- FOR EXAMPLE, to modify the log of an already commited revision:
svn propedit --revprop -r <REV_NUM>
svn:log <URL_OF_REPOSITORY>
OR: svn propedit --revprop -r <REV_NUM>
svn:log (if already checked out copy in curr. dir)
Using LD, the GNU linker (also note LDD for finding the library
dependencies of an existing binary (executable or shared object;
LD can be downloaded as part of the
binutils distribution))
Static versus dynamic linking
-- As of GLIBC-2.17, GLIBC deprecated
the use of dlopen in dynamic linking. As a side effect, they have
deprecated calls to gethostbyname, getaddrinfo, getpwname, etc.,
for all statically linked executables. The argument seems to be
that nsswitch needs to decide at runtime which libraries to load
to resolve addresses, and that a local site may add a custom
dynamically linked library to resolve addresses in a custom mannter.
They further argue that the site may use a newer (or older) version
of a dynamically linked library not available when a statically
linked executable was created.
The argument then continues that a statically linked executable
cannot support this local, dynamically linked library for
resolving custom addresses, and so statically linked executables
should not be allowed to call gethostbyname, getaddrinfo,
getpwname, etc. Therefore glibc is removing those functions
from its static libc.a. In fact, glibc still allows this, but
only by creating a custom libc.a
configured with --enable-static-nss,
and they do not provide that support in the default libc.a
provided by most Linux distros. The flag causes libc.a
to call dlopen to load any local, dynamically linked
libraries (custom or otherwise). A second alternative
is to compile with the statically linked libc.a of musl. And finally,
I grew so annoyed, that I created my own third alternative.
- gethostbyname_static.tar.gz:
(a definition of gethostbyname/getaddrinfo that can be statically
linked into your executable; the code forks a child, which execs
into a dynamically linked program, which can then call gethostbyname
or getaddrinfo dynamically, and report back the results to the
statically linked parent).
-
Static Linking Considered Harmful (the argument to forbid
all static linking, by Ulrich Drepper)
-
The problem with nsswitch (an important counter-argument
to the argument above, on security grounds)
- nsss
(nss library for getpwnam, but not the network address resolution)
-
a counter-argument couched in the language of the vernacular
- And of course, a statically linked executable could fork a
dynamically linked proxy that calls gethostbyname,
getaddrinfo, getpwname, etc. It would be slower, but most
statically linked executables make these calls infrequently.
The tension is then whether:
(i) to support a site's local, custom dynamically linked library for
address resolution with a proxy (slower address resolution),
dlopen (deprecated by glibc), or musl (it works); or else
(ii) to forbid statically linked executables from using
address resolution and therefore create barriers to network access.
Executable and Linkable Format (ELF)
by UNIX Systems Laboratories, as part of the Application
Binary Interface (ABI)
ELF, symbol versioning,
thread-local storage, etc. (by Ulrich Drepper; click on "ELF")
A Deep dive into (implicit) Thread Local Storage
(thread-local storage (TLS) and thead control block (TCB) and pthread_t; Excellent!)
All About Thread-Local Storage
Versioning a structure in glibc (including dynamic versioning for
the static glibc.a library)
-
NOTE:
sym = dlsym(RTLD_NEXT, "pthread_cond_signal");
breaks under versioning, and why.
- ELF Tutorial
(from OWDev Wiki; good
explanation of symbol relocation for statically linked ELF)
- Relocation Sections
(for statically linked ELF: etc., from Oracle Linker and Libraries guide;
includes x86-specific relocations; see x64 for x86_64 relocation types)
-
Load-time relocation of shared libraries
- What Are Tentative Symbols? (explanation of common symbols: SHN_COMMON, STT_COMMON)
-
Weak undefined symbols and dynamic libraries
Why weak symbols are broken for non-PIC executables. The following
works only if your base executable was compiled with '-fPIC'.
__attribute__((weak)) void weakfn(void) {}
int main() { if (weakfn) weakfn(); }
- Learning Linux Binary Analysis
(O'Reilly Safari: Ryan O'Neill: ELF internals for reverse engineering
and forensic analysis)(
-
Position Independent Code (PIC) in shared libraries
- ELF dynamic linking (excellent general overview, even if intended for PowerPC)
-
EresiArticles (great information on ELF at bottom of page)
-
Program Library HOWTO
-
ELF.doc.tar.gz or
ELF.txt
-
Ian Lance Taylor articles on linkers (excellent!)
(or alt)
- eh_frame
(exceptions and unwinding the stack, by Ian Lance Taylor)
(and
eh_frame_hdr)
"The old way this worked was that gcc would create a global
constructor which called the function __register_frame_info,
passing a pointer to the .eh_frame data and a pointer to the
object. The latter pointer would indicate the shared library,
and was used to deregister the information after a dlclose. When
looking for an FDE, the unwinder would walk through the registered
frames, and sort them. Then it would use the sorted list to find
the desired FDE."
"The old way still works, but these days, at least on GNU/Linux,
the sorting is done at link time,"
-
The libunwind project (debugging, exception handling, introspection)
-
Startup state of a Linux/i386 ELF binary
- Linux x86 Program Start Up (or "How the heck do we get to main()?)
by Patrick Horgan
- Linkers and Loaders
by John Levine
-
The ELF Specification
-
PLT and GOT - the key to code-sharing and dynamic libraries
-
All About Procedure Linkage Table (PLT)
- Calling Global Cpmstructors
(osdev.org: How do the crt*.o files invoke constructors?)
-
Dynamic Linking (from
Computer Science from the Bottom Up, by Ian Wienand)
- The Art of ELF; Amalysis and Exploitations
(FIUxIuS' Blog)
- Abusing .CTORS and .DTORS For FUN and PROFIT
(by Izik)
-
DSO (Dynamic Shared Object) Howto (by Drepper)
- Redirecting functions in shared ELF libraries
-
ELF statifier (Convert executable with dynamic libraries
into self-contained library; program can then be copied
to any compatible architecture, since it no longer uses
dynamic libraries);
See also ldd: List Dynamic Dependencies
See also dlopen: Dynamic Library Open
-
libelf (See "man elf" and
Sun ELF tutorial)
- libbfd (part of binutils??), Binary File Descriptor handles any
binary format, not just ELF. Try `info bfd' for docs.
-
ELF symbol versioning docs
- LSB 3.1 Specification (ELF)
-
ELF Kickers (small example programs dealing with ELF)
Linux Memory Management Frequently Asked Questions
PaX (Wikipedia article) (and
PaX home page (Linux security team implementing address
space randomization and other techniques))
Project Zero (blogs from Google's Project Zero team)
Debugging with GDB (including internals manual)
rr-project: lightweight recording and
deterministic debugging (reversible debugging based on GDB)
QIRA ("QIRA is a timeless debugger.
All state is tracked while a program is running, so you can debug
in the past.")
gdbgui
(A browser-based frontend to GDB (GNU debugger))
Modular visual interface for GDB in Python
DDD - Data Display Debugger
(graphical front-end for gdb and other debuggers)
Process-in-Process
software (and modified GDB to debug two processes in one address
space)
Radare (and Radare2)
(Debugger/disassembler influenced by
IDA/IDA Pro/Hex-Rays)
HopperApp (another tool for reverse engineering)
ELF binaries on Linux understanding and Analysis (with Tools for Binary Analysis)
Linux x86 Program Start Up or - How the heck do we get to main()?
by Patrick Horgan
GNU gprof (see also
gcov)
Using as (gas - GNU assembler)
- For assembly with
source file listing, try
"
gcc -c -g -Wa,-alh,-L file.c > file.s", or
"gcc -c -g -Wa,-ahls=file.s file.c"
(man as:
-al: emit assembly listing, -ah: emit high level C/C++ source;
-L: keep local symbols; NOTE: "-g" is required, and "-S" is purposely
omitted, since high level C/C++ source is apparently emitted only
as part of full compilation)
- For all predefined and file-defined macros, try:
cpp -dM myfile.h or cpp -dM < /dev/null
- CGEN
(generator of assemblers, disassemblers and simulators,
including IA-32, SPARC, etc.)
-
TRIPS (nanoscale microprocessor chips to reduce on-chip
latency and to provide greater power efficiency; dataflow-style
computing in hardware)
LIB BFD, the Binary File Descriptor Library
libtool (for managing shared libraries,
w/ software)
GCC (gcc) online documentation
(also see assembly
with source file listing)
GNU C Compiler Internals (Wikibook)
G++ Internals (e.g. name mangling algorithm, etc.;
Note c++filt from binutils is available for C++ demangling))
C++0x Support in GCC
(with links to the issues discussed in the C++0x standard)
lint and cpp are also bundled into gcc;
Try "gcc -Wall -Wextra", "gcc -pedantic" for lint, and "gcc -E" for cpp
(and see "gcc -traditional-cpp"); What about /*NOTREACHED*/, etc.?;
Try "cpp -dM ANY_FILE" to see predefined macros.
indent (some other C/C++ beautifiers are:
GC GreatCode, artistic
style, and the old UNIX cb)
GMP (GNU MP, or Multiple Precision library)
GNU Autoconf
-
Autotools Mythbuster
-
GNU make
- ccache manual
(speeds up recompilation by caching earlier results)
-
Bison (and
flex (a scanner generator) -- patterned after yacc and lex )
-
GNU Emacs manual
- vim
-
GNU TexInfo
-
GNUshogi
- GNU Go (game)
-
RFC for NFS Version 4 (also see
paper on NFS 4)
- Midnight Commander File
Manager
(can also look inside tar, rpm and other archive formats)
-
Scientific Applications on Linux (SAL)
- GSL - GNU Scientific
Library (includes special functions, linear algebra (and BLAS
support), integration, differential equation solver, permutations,
combinations, FFTs, histograms, interpolation and extrapolation,
minimization, root-finding, least squares fits, physical
constants, etc.; also, see
GSL manual)
- GNU R Project
(Statistics and graphics project, good graphing environment)
-
TeXmacs (scientific text editor, plotting, spreadsheet, etc.;
can be combined with popular symbolic algebra software)
-
Mathemagix (C++ extension for symbolic algebra, numerical analysis;
with its own interpreter)
- Gnuplot Central
(not at GNU site)
-
The Plotutils Package
- SLxfig (excellent,
part of S-Lang, not GNU)
- If xfig produces Type 3 font, do: Inkscape and exporting it
to PDF with the export option "convert text to paths"; Then add
"\pdfminorversion=4" for pdflatex.
- Flowchart Maker and Online
Diagram Software
-
Jgraph (simpler than gnuplot, supports bar graphs, also
distributed from Debian and SAL; But also try Python pkg, matplotlib)
- xfig, fig and
associated software
(also xfig manual)
- Diagrams.net
- Creating diagrams in ASCII
-
Bash Reference manual (Bourne-Again Shell)
-
GNU Wget Manual (Copy tree of URL's to local disk)
- byobu (nice
multi-panel interface for
GNU screen)
- tmux (terminal
multiplexer) (an alternative to screen)
-
GNU maxima (no html manual?)
- readline
- rlwrap (adds readline line editing, persistent history,
keyword/variable/filename completion, around any command)
- Open Source licenses
(GPL, LGPL, BSD, etc.)
-
GNU Lesser General Public License
and
GNU Library GPL (less restrictive)
(public copyright for linking a library to an application)
-
Creative Commons License (non-commercial, sharable text)
- Cygwin
(Port of GNU gcc, bash, and related utilities to Windows;
Includes API interface for Windows)
- SEE ALSO: MKS/Windows, teraterm for Windows,
Linux
environment from S. Bratus's old home page
-
putty download page (and pscp, ssh clients for Windows)
- Mosh
(mobile shell: novel ssh utility; keeps connections while roaming;
fixes other terminal emulator bugs; e.g., uses on the standard UTF-8)
- Perl Home Page (also see
CPAN, Comprehensive
Perl Archive Network; not GNU, but they should be)
- Python
(with
Python Documentation)
-
pip (and pip3) user installs of Python packages
(e.g.,
pip3 install --user MYPKG)
-
Python3 Tutorial: History and Philosophy of Python (Excellent!)
-
Python tutorial (nice and short, well-organized:
Byte of Python by Swaroop C H)
(or alt)
-
Rosetta Code Wiki for Python (Simple programs done in
Python and other languages, for comparison)
- ZenRows Blog -
How to Scrape JavaScript-Rendered Web Pages with Python
- Data Science Skills: Web scraping javascript using python
- Nullege: A Search Engine for Python
source code (good for finding examples of Python code)
- Python 2.4 Quick Reference Card
(alt)
(pdf, open-office; 18~pages)
-
Python 2.6 Quick Reference
(or alt)
(50&nbs;pages but could print 2 pages per side)
-
Python mechanize package
-
Selenium with Python
- pygame - Python game development
-
Python for Lisp Programmers (by Peter Norvig)
- Python is Not C: Take Two
(optimizing Python performance)
-
PerformancePython (compares NumPy, weave.blitz, weave.inline,
f2py, Pyrex/Cython, Matlab, Octave, C++: shows blitz through Pyrex
to be "close enough" to C/C++ in speed)
- Pythran
- Numba (annotate Python numerical
code, and let it infer types and compile natively)
-
Numba: A High Performance Python Compiler
(part of pydata project??)
-
Pythran (compiles numeric Python code to templatized C++,
which uses multicore and SIMD instructions, where the hardware
supports it)
- PyPy
(fast, compliant version of Python with Just-in-Time compiler)
- Cython: C-Extensions for Python
(mostly compatible with Pyrex; large compatibility with Python;
Generates C code that can be inspected by a human being;
Optional type declarations for better code generation)
-
Stackless Python (userland, green threads, coroutines)
- GDB support for debugging
Python and CPython
- Numba (just-in-time compiling
for Python and NumPy using LLVM)
- Django
(Python Web framework for web development)
- Python and SWIG
(calling Python from SWIG)
- IPython
- Debugging: ipdb or
from IPython.core.debugger import Pdb;
Pdb().set_trace()
-
Pyrex (compiler for Python -- large compatibility, subsumed
by newer projects)
- eyeD3 (Python MP3 support)
-
Dive Into Python (advanced book: dictionary (association lists),
regular expressions, lists, filtering lists, lambda, objects, etc.)
-
Foreign Function Library for Python
-
IDLE (GUI-based Integrated Development Environment for Python)
- unlade-swallow (speeding up
CPython)
(or
alt)
-
Python examples (organized by topic)
- SciPy
(Matlab-like functionality: graphing, plotting, optimization,
special functions,
genetic algorithms, linear algebra, ODE solvers),
NumPy
(for speeding up array operations, matrix manipulations, stats),
Twisted
(event-driven networking framework: à la wrapper around
select system call),
PyRex (mixes Python and C data types, and copiles code
into C extension module for speed),
server.py (networking),
userball.py (games),
wxPython (GUIs, web interfaces),
bounce.py (visualization),
Talks at PyCon that teach how to be better programmers
-
Tutorial from Python docs
- Thinking in
Python
(or alt)
(a more advanced book and still incomplete, but good expository style)
-
GNU Hurd Reference Manual (UNIX Kernel replacement
supporting Mach microkernel)
-
LinuxThreads library (Note that Linux currently uses
NPTL (New Posix Threads Library), and not LinuxThreads
implementation of pthreads.)
-
LD_ASSUME_KERNEL (and how it chooses between
LinuxThreads (in /lib or /lib/i686) and NPTL (in /lib/tls).
-
GNU Portable Threads (GNU Pth)
(non-preemptive user space threads; optional API for pthreads;
does _not_ take advantage of multiple processors)
> non-GNU pthreads packages follow:
- Subtle issues when using POSIX pthreads:
- Use volatile keywork if one thread sets a global variable
in a tight loop, and you want another thread to read it.
-
ABA problem (a tagged state reference often provides a simple
solution, but that requires a double-word compare-and-swap)
-
C++ and the Perils of Double-Checked Locking
(nice description from Dr. Dobbs of the DCLP problem)
-
The Foundations for Scalable Multi-Core Software in Intel Threading Building Blocks (TBB: Intel Technology Journal)
- Linux ptrace
documentation (to be revised and added to 'man ptrace')
-
Status of upcoming extension to Linux 'man ptrace'
(as of Feb. 13, 2012)
-
Example of ptrace usage
-
Tracing tricks with ptrace
-
epoll is not O(1)
- NOT NECESSARILY GNU, BUT SHOULD BE
- ouah.org (Security research,
forensics, and intrusion detection; not active since 2006??)
- MIT CSAIL Computer Systems
Security Group (led by Nickolai Zeldovich)
- Analysis of some well-known malware:
- PGP
- Open Group portal
-
The GNU C Library - Table of Contents
(includes information on POSIX, BSD, SYSV and other standards)
- Internet Software
Consortium (ISC) (nonprofit corporation for
"reference implementations of core Internet protocols",
including
code useful for developers)
- Octave Home Page
(Matlab-like language -- GPL;
and
free Octave book)
- Scilab Home Page
(Another Matlab-like language -- GPL)
-
Internet Software Consortium (memory leak)
(including
libefence (electric fence), search on it from this page.)
-
electric fence 2.2 (libefence) (provided by kallisti5)
- mcheck() for GLIBC
- mallopt() for GLIBC (adjust parameters that control the behavior of malloc)
- GLIBC malloc tunables
(e.g., glibc.malloc.mmap_threshold; After glibc-2.34, malloc debugging moved to libc_malloc_debug.so: LD_PRELOAD=/lib/x86_64-linux-gnu/libc_malloc_debug.so.0 MALLOC_CHECK_=5 and MALLOC_PERTURB_=1; and MALLOC_CHECK_=XX (< 4) will tolerate malloc bugs)
- DUMA library
- DUMA library
(Detect Unintended Memory Access;
newer replacement for Electric Fence)
-
Heap Layers (Emery Berger et al.,
also see Hoard)
-
tcmalloc: Thread-Caching Malloc (google-perftools)
(and
alt)
-
LeakTracer - trace and analyze memory leaks in C++ programs
-
Cooperative Bug Isolation Project (automatically getting end-user
feedback)
- UMLGraph
(declarative specification of UML diagrams)
- Open Source Hosting
- Hosted Apps - sourceforge
- Freshmeat.net
(open software development, pointers to open source projects
with ratings)
-
FreeSoftware.com (open source software, and meta directory)
-
DiskSim Simulation Environment
- cacheprof
(simulates cache of one's choice, and produces annotated
source code about cache misses, plus per-procedure summaries)
- CPU-World
(full specs for current and older CPUs)
-
Calibrator (measures cache, TLB, latencies, etc.)
-
OProfile (Linux kernel module, enabling system-wide
low-overhead profiling, without wrapper, etc.)
- Rightmark Memory Analyzer
(reports on cache, TLB, CPU model, etc.: assoc., num cache blocks, etc.)
- strace,
truss (based on ptrace system call): not part of GNU,
but it should be
-
NPTL Trace Tool Project (sourceforge)
-
linux-insides (EXCELLENT: book from booting to everything else)
- System calls in the Linux kernel. Part 3.
(vsyscalls and vDSO)
- Doxygen
(generates code browser with links for C++, C, Java and others;
generates HTML or latex,
with description)
- Markdown
(wiki-like readable syntax, converted to HTML)
-
MEditor.md: Open source online Markdown editor
-
Sphinx tutorial (but use python3-sphinx package of Linux distro,
instead of Python venv (virtual environment))
- Diátaxis: A systematic approach to technical documentation authoring
(excellent!)
-
Linux source code (bootlin)
-
Linux source code (docs.huihoo.com: doxygen)
-
Linux source code search (livegrep.com)
- LXR
(code browser; SourceForge: Project Info - LXR Cross Referencer)
- sourceware.org
(web CVS access to many software projects: gcc, glibc, ...)
- Bonsai
Bonsai -- the art of effectively controlling trees
(CVS web interface
(see intro),
with "cvsblame" to read CVS comments about why lines were changed)
-
MIT distribution site for PGP
- TTH
(TeX to HTML, excellent at converting TeX and LaTeX math)
Productivity Stuff for text manipulation
- (pnm is "lingua franca" for UNIX/Linux graphics);
see
Webslide for webifying postscript files
- html2ps
- Others:
-
Ghostscript, Ghostview and GSview (gv)
(Aladdin Ghostscript provides postscript to pdf translation)
-
Apprentice Alf's Blog: Everything you ever wanted to know about
DRM and ebooks, but were afraid to ask
- Calibre - E-book management
(ebook-convert online (pdf, MOBI, etc.); free download of calibre available)
- mpack/munpack
- Mutt.org (small, powerful mail client)
- sup (next step after mutt;
indexes and tags e-mails without setting up folders)
- Mailinator
(free throwaway temporary e-mail accounts with no verification needed)
- Temp Mail
(free throwaway temporary e-mail accounts with no verification needed)
-
HowTo: Block/Disable/Bypass Online Surveys
- zoo.tgz and other files
makeinfo (from texinfo to info),
texi2dvi,
texi2html,
dvips (and dvi2fax?)
(IMPORTANT: Use "dvips -P cmz -t letter" to avoid
fuzzy fonts in pdf),
dvi2tty (to text),
ps2ps (Ghostscript PostScript "distiller", simplifies postscript
into something more likely to print correctly.)
latex2html
(see also TTH (TeX-To-Html)),
pdftex and pdflatex (part of tetex distribution, bypass font issues
of converting tex->dvi->postscript->pdf),
rvterm (sp?),
tgif (for 2-D drawing and
image manip.: e.g.: gif -&get; vector drawing),
jpeg2ps,
also see
pointers to other tools
- Image restoration and enhancement
Intro talks about
Linux, LaTeX, and other UNIX tools.
Linux ( Linux Home Page)
-
Linux FAQ
- SEE ESPECIALLY list of Linux online books;
for example: Rute User's Tutorial and Exposition
-
Linux Unleashed (book with good introduction)
-
The Linux Documentation Project Works (all LDP docs)
-
Linux Articles (O'Reilly Network)
-
LDP HOWTO index
-
The Linux Documentation Project: Guides (LDP Guides)
-
The Linux Documentation Project: FAQs (LDP FAQs)
-
Single list of HOWTO's
- Linux HOWTO's (old?)
(help doc's on "installing Linux, Installing mail, etc.")
UNetbootln (Added to Fat32 USB key;
Installs Linux .iso as live system from USB)
-
Linux Hardware Compatibility HOWTO
(or
alt)
- New Seagate drives to not support Linux or Mac
(and Ubuntu reports problems with NVIDIA graphics cards,
as of Jan., 2008)
- Linux Hardware Database
(alt URL),
(contains many models, be sure to also click on user "reports")
- Guide to IP Layer Network
Administration with Linux
(includes IP (layer 3), iproute2, ip tunnel, tcng (Traffic
Control Next Generation),
traffic control (shaping, scheduling, dropping),
etc.)
-
Scanners, still cameras, video cameras, etc.
-
Captive by Jan Kratochvil (uses Windows's own ntfs driver
for read/write access to NTFS)
- Linux NTFS Project
(Linux does read/write to FAT16/32, but mostly read for NTFS;
but also see here; For documentation of NTFS internals, see
here)
-
LinuxPrinting.org
(also look at
LinuxPrinting.org Printer Database to find your printer, and
then choose appropriate driver)
-
FAQ on Linux NTFS
(question 3.2 is on software to let Windows read Linux ext2/ext3)
-
Linux Wireless HOWTO (WiFi and networking)
-
Using a RADIUS Server on Ubuntu 14.04 for WIFI Authentication
(certificates)
- parprouted (allows bridging between Ethernet and wireless network)
- Modem-HOWTO
- cell phone as USB modem
(requires USB Data Cable)
-
Xen (paravirtualization (modified guest OS)
like Qemu, but operates using a microkernel to host,
checkpoint, and even migrate guest O/S's; supports Linux,
but not Windows, and may be harder to use than Qemu;
See also
example of using Xen to test Debian upgrade)
- User Mode Linux
(UML; needs no kernel modification)
- VMware
(with a choice of
virtual appliances (guest O/S's) to download:
enter in "search appliances")
- VirtualBox
(free x86 virtualization originally developed at Sun;
native virtualization (e.g. 32-bit Intel OS on 32-bit Intel hardware);
copy-on-write based snapshots, available on Mac Leopard 10.5)
- Page faults in user space
(making it easy to do live migration, e.g., with a DMTCP plugin)
(with an RFC)
- QEMU
(similar to Bochs, but requires IA-32 architecture, higher speed;
includes free O/S zoo of other O/S images prebuilt to run under
Qemu; included in Debian)
- KVM - Main Page
- LXC - Linux Containers
- Docker - the Linux container engine
- Apptainer container
(related to Singularity containers)
- LXD (containers including
systemd)
-
Understanding and Hardening Linux Containers
- Asylo framework
(security in multi-tenant cloud using Intel TXT and SGX:
see here)
- A Container for HPC : ADMIN Magazine
-
Red Hat OpenShift 3 : le PaaS privé avec Docker
(in French, in case you hadn't guessed it, but a really nice article)
- Flockbox
(
"self-contained minimal VM that lets you use LXC")
- (modified ChromeOS (minimal Linux
kernel) that runs software in containers; can be booted
with PXE
(e.g., IPXE))
-
Amazon Bottlerocket FAQ (low-resource footprint, for faster boot times,
smaller attack surface, automatic OS updates)
-
Blue Pill (Wikipedia, thin hypervisor)
- Bochs
(emulate another Intel IA-32 based PC O/S in a window in the current
one, à la VMware, VirtualPC, etc.; also handles
x86-64)
- lguest
(paravirtualization, small (5000 lines), kernel module)
- KVM
(KVM now in Linux kernel; support for virtualization
using newer virtualization support of Intel/AMD)
-
Plex86 (similar to Bochs, but requires IA-32 architecture,
higher speed, see also
Lightweight Plex86)
- ReactOS
(free Windows NT emulator, which could run, for example, on top
of Bochs in a Linux window, or in its own partition)
- MinGW (build Windows apps
natively or cross-compile from Linux; provides a shell for
Windows, and can use Gtk or Qt cross-platform GUI libraries)
- Wine HQ (run Windows
applications as windows directly in Linux)
- Open vSwitch
- Linux Wiki
(via wikia.com)
- Linux Knowledge Base
-
Ghostscript output devices (Linux printer drivers,
with
printer compatibility)
- Linux Ghostscript printer compatibility list
-
Digital Camera Setup (Waikato Linux Users Group)
-
BSD 4.4 book by McKusick, Bostic, Karels, Quarterman
(Chapter 2, Design Overview)
-
Linux source code (bootlin)
- FreeBSD and Linux kernel source code
(via LXR)
- alt. Linux kernel source code
(with alt)
-
Linux Device Drivers (book)
- KernelNewbiew.org
(with stuff about the Linux kernel)
-
Advanced Linux Programming (with IPC, threads, etc.;
book is also available as free download from Web page)
-
Linux from Scratch (LFS) (online book, tells how to download
and install Linux pieces)
- Linux vendors (and sources of free contrib software)
- Linux compatibility (The general rule is watch out for shortcuts
that vendors use to replace hardware by Windows software drivers.)
-
General hardware compatibility (Linux HOWTO)
-
Linux on Laptops (setup for different models)
- Supported graphics cards (Xfree86)
- No WinModem, SoftModem, Windows printers, etc.
(these usually depend on proprietary Windows-based software drivers)
(search for Linux Modem Compatibility Database)
- true 3-button mouse (not emulated with Windows driver) ---
(2-button mice can be used, but you need to simultaneously
press both buttons to emulate the middle button for X-windows)
- sound card (for example, avoid DSP-based "integrated" sound using
CPU and Windows-based software driver)
-
Linux Information
(including
man pages)
-
Linux Administration Made Easy
-
TrinityOS: A Guide to Configuring Your Linux Server
-
Linux + Windows HOWTO
- LINUXISO.org
(Copies of CD ISO images for most Linux distributions)
See:
- NOTE: If starting with a Windows computer, you will
need to repartition, and resize the ntfs partition. (An alternative
is "Wubi: Ubuntu inside Windows" (see below).) To do it by
partitioning with free software,
ntfresize 1.9.0 or later is recommended. This is included
in Debian Sarge, Mandrake 10.1, and many other distributions
(see previous link).
- Alternatively, one can avoid repartitioning by using the Wubi Ubuntu installer to install Ubuntu inside Windows.
This makes it easy to remove this Linux (using add/remove from the control
panel), but if Windows fails to work, Ubuntu may also fail.
- Gentoo Linux
- Alpine Linux
- NixOS
(functional package system --- multiple versions can coexist)
- AndLinux (Uses Linux kernel
ported to Windows)
- Arch Linux
(simple, easy to tweak from the command line)
- Debian Linux
- Debian Manuals
(and Debian Documentation)
-
Ubuntu Linux (based on Debian, but easier install/configuration,
includes recent versions of packages; product of company, but
freely available; fairly popular choice of Linux;
good
Wiki documentation)
- HowtoForge (Linux)
(Select distro and search on "Perfect Desktop")
-
The best netbook-friendly Linux distros (excellent reviews, as of June, 2009)
-
Ubuntu Compatibility -- Hardware Support/Machines/Netbooks
-
UNetbootin (add to USB key in standard fat32 format;
In BIOS, list USB Flash drive first in boot order;
then download and install most Linux live .iso CDs; does not overwrite
existing files; does not work with Mandriva 9 and Mandriva 10)
- Lubuntu (lightweight Ubuntu)
(fully compatible full-featured Ubuntu, but using LXDE lightweight
desktop, and other choices for ligher resource usage)
- Puppy Linux
(uses lightweight app's to fit mostly in RAM, can run fast
from USB key)
- Linux Mint
(includes browser plugins, media codecs, DVD playback support;
without having to separately load those packages, etc.)
- Note: Linux Mint MATE for lowest RAM; then Linux Mint Xfce; then others
- Moblin
(Intel-sponsored Gnome-based Linux for Netbooks, using
Suse yum package manager; doesn't currently work on non-SSSE3
CPUs like AMD??)
- Jolicloud (Linux optimized
for netbooks, with strong driver support)
- Aurora (Ubuntu for the Asus EEE PC
and others, formerly EEEbuntu))
- Ubuntu on EEE PC (Wiki,
somewhat dated?)
- DebianEeePC
- Linux4One (Ubuntu for Acer Aspire One)
- Google ARM-based Chromebook
- Acer C7 Chromebook (2 GB RAM, 320 GB disk, USB, HDMI,
VGA, SD card, 11.6" screen, 4-cell battery, no Ethernet conn.,
$200, 11/2012)
- 8 Chromebook Tricks You Need to Know
- Samsung Series 5
Chromebook (how to enter developer mode)
-
dnschneid/crouton Wiki (running Ubuntu in VM-like environment
(actually, chroot) under Chrome OS)
-
How to dual-boot Ubuntu Linux on your Chrome OS device
- Chromebook as UPnP media server
(Universal Plug and Play)
- Blogs by Ville Timonen
(Linux initramfs and iSCIS, ramndom file read benchmark, and other
weird stuff)
- Chrome Remote Desktop App (like VNC, includes remote audio feed
(for Linux/Windows/Mac OSX, runs in Chrome browser))
-
noVNC (VNC running in the web browser, for HTML5)
- VP9 (Wikipedia)
(open, royalty-free video compression from Google: for
Chrome and Firefox browsers)
- Ubuntu packages
- Ubuntu package search, maverick, i386
- Mirrors of Linux distro package
repositories
- Wubi - Ubuntu Installer for Windows (installs/uninstalls as if it were a Windows application, no disk partitioning)
- Boot and run Linux from a USB flash
memory stick / USB Pen DriveLinux (compatible with
many dialects of Linux)
- Fedora/Redhat Linux
- The LiveCD List
- Sabayon Linux
(based on Gentoo, but also w/ binary packages;
integrated well-thought out Desktop
using Beryl)
- Damn Small Linux
(50 MB, on business card CD, based on Knoppix)
- Feather Linux
(64 MB, on business card CD, based on Knoppix;
alt)
- Helix
(data recovery, computer forensics, vulnerability analysis)
- Knoppix Linux (LiveCD,
runs directly from bootable CD-ROM)
-
Kanotix (Variation of Knoppix; easy install to hard disk;
includes latex (tetex);
monthly updates based on latest "Debian unstable"; after install
to hard disk, use dselect, etc., to continue to update from web)
- Morphix (LiveCD, easily installable to
hard disk; modules making it easy for many others
to create derivative versions)
- Elive (Live CD,
shows off Enlightenment desktop manager, eye candy)
- Gentoo Linux
- Alpine Linux
- Mandriva (formerly Mandrake;
see also Mandrake Move LiveCD)
- MythTV
(Linux-based computer replacement for recording/program guide/home
theater, etc.)
- OpenSolaris CDs (OpenSolaris kernel instead of Linux kernel, but
combines Solaris features (e.g. dtrace), SMF, Zones with Linux/GNU
utilities)
- BeleniX
(OpenSolaris with some added GNU utilies)
- Nexenta
(OpenSolaris kernel and runtime libraries
with Debian GNU package system on top of it)
- SEE ALSO Linux versions of Windows software
- SEE ALSO Linux Tutorials
for end users (and many other
Linux portals)
- Linuxbegin.ru
is a great site for Linux beginners (but in Russian)
- Packages for Linux
- Debian (.deb) and .rpm ar the two most common package systems.
- A third alternative is Klik, which bundles all dependent packages,
using more disk space, but avoiding dependency issues.
- Autopackage (The name says it all.)
- RPM (originally Redhat Package Mangement)
- To translate from FILE to rpm package containing FILE:
rpm -qf FILE
- To find the package or debuginfo package in a repo, use:
RPM Search
(sorted alphabetically by package name, including version;
"simple search" finds package from filename; "advanced search"
finds package for your distro, from packagename)
(If you're lucky, debuginfo-install will find this already in your
vendor's repo. Otherwise, hope that some kind person has already
pre-compiled the debuginfo package, usually as PKG-debuginfo-VERSION.rpm.)
- www.rpm.org
(repository of packages for Linux, checking dependencies; To unpack
or extract manually, do:
rpm2cpio emacs-common-21.4-20.el5.x86_64.rpm | cpio -idv
- To build an rpm:
yum install rpmdevtools
rpmdev-setuptree
# .rpmmacros and ~/rpmbuild now exist
yumdownloader <SOURCE_PACKAGE>
# (or else just download the .src.rpm file)
rpm -i <FILE.src.rpm> # source rpm (may be .srpm)
# Probably look at NAME.spec, search on -O2, and uncomment optimizations
rpmbuild -ba ~/rpmbuild/SPECS/NAME.spec
- In OpenSure, to build a glibc binary _with_ debug symbols, see
this post.
- If you see the message
warning: user mockbuild does not exist - using root, try:
yum install mock
useradd -s /sbin/nologin mockbuild
and try rpm -i again.
- Debuginfo for .rpm and .deb packages:
-
How to create an RPM package
- RPM Packaging Guid
- After running rpmbuild -ba SPECS/.spec, look at
/var/tmp/rpm-tmp.* for the actual script that was executed
by rpmbuild during testing (%check):
sh -e /var/tmp/rpm-tmp.XXXX
You can then re-run this for further in situ testing.
-
Common Rpmlint Issues - FedoraProject (package: rpmlint)
- mock and koji are useful Fedora packages for
testing dependencies, and doing builds of foreign rpms, respectively.
-
Packaging Guidelines - FedoraProject
- RPM Find.net
- RPM Search
- RPM Livna.org
Repository of additional packages, e.g.: mplayer, xine, videolan-client,
xmms-mp3; whose licenses may be problematic for free software
-
How to create an RPM package - FedoraProject
- RPM Packaging Overview
- Cygwin
(Linux utilities operating directly under Windows;
see also
Cygwin User's Guide)
-
Damn Small Linux
-
Knoppix Linux (LiveCD)
(w/
ISO images and w/
Knoppix tutorial in German)
- Morphix (with pointers to derivatives)
(robust package system -- especially for building from source;
excellent choice when you want to frequently get the latest version
and not have to do a Linux distribution upgrades)
- Debian
(Excellent technical documentation)
-
Debian Linux on Averatec 3200
-
Downloading Debian CD/DVD images
-
Debian GNU/Linux Desktop Survival Guide
- Debian Documentation
(and More docs)
-
Debian installation instructions
(or
text version)
- Debian FAQ
(extensive, review first)
-
Debian New Maintainers' Guide (How to write Debian .deb packages;
from DDP
Developers' Manuals)
- Debian post-install reference
(with testing version)
- Wajig
(simplifies some common Debian system administration tasks)
-
Debian packages (list, search, view contents, etc.;
note also non-free, Contrib, etc., that are not on Debian CDs,
but are available)
- Debian GNU/Linux
Hardware Compatibility List
- NOTE: Use dpkg --search FULL_PATHNAME to find out
what package a file comes from.
- NOTE: (i)
apt-get source PKG to get source;
(ii) dpkg-source -x FILE.dsc to unpack
Debian source packages; (iii) cd into PKG;
and (iv) dpkg-buildpackage -rfakeroot --usigned-source -b
to build it without installing.
- How to use debug libraries on Ubuntu
(using /etc/apt/sources.list.d/ddebs.list for -dbgsym
packages; being considered by Debian for adoption)
- Find package name from filename: sudo apt-get install dlocate
-
FAQ - Basics of the Debian package management system
(from
The Debian GNU/Linux FAQ)
-
Some questions and answers on Debian
- One recommends: "apt-get install discover hotplug aptitude"
discover - PCI device detection, hotplug - USB hotplug support,
aptitude - a better package management software
- see also apt-cache: apt-cache search keyword1 keyword2 ...;
apt-cache show pkg1_name pkg2_name ...
-
Debian CD images (suitable for CD burners)
-
Debian floppy boot and kernel images
- Debian documentation
- http://src.openresources.com/debian/
(Debian individual source files in html)
-
DDP Developers' Manuals (for authoring Debian packages,
start with
New Maintainers' Guide)
-
Mandriva Linux (formerly Mandrake; uses RPM packages)
- Packaging guildelines for different Linux distributions
- Red Hat Software, Inc.
(commercial, only)
(see Fedora
Project,
or Fedora Home Page,
or LinuxInstall.org
(also for Fedora))
- RPM Livna.org
(packages with problematic licensing, for use with Fedora/Red Hat)
RPMfind
-
CentOS rpm's
-
CentOS Community Build Service (CBM) for rpm's (contrib packages)
- www.rpm.org
-
Pre-configured Linux PC's
- Linux man pages
-
Linux (SLUG, good Linux help features)
- The Linux Kernel Archives
Productivity Stuff for Linux
SEE ALSO Application equivalents - openSUSE
SEE ALSO Linux versions of Windows software
-
Sleuth Kit and Autopsy (forensic tools and disk recovery
for FAT32, Linux, etc.)
- Clara OCR
-
CMU Sphinx project (speech recognition)
- MBROLA Project
(speech synthesizer, with many
text-to-speech frontends
- HylaFAX
- Efax-gtk
-
Scientific graphs (including interactively)
- Graphviz
(easy building of complex graphs and diagrams)
- graph-tool
(generate and analyze graphs; also draws graphs or interfaces w/ graphviz)
- Matplotlib Documentation (Matlab-style plotting)
- Plotly Python Graphing Library
(interactively changing graph display)
- http://d3js.org/">D3.js - Data-Driven Documents
(interactive derivation of graphics using HTML standards)
- CodeLite Open Source C/C++ IDE
(analog of Microsoft Visual Studio)
-
VauCanSon-G LaTeX package
(easy building of complex automata with simple commands)
-
Asymptote (vector graphics language for easily
programming 2-D and 3-D diagrams)
- hsqldb SQL engine with
JDBC driver; or MySQL
with free open source
downloads available
(and see SQirreL SQL Client)
- For VMware, Virtual PC, see
Bochs
and Plex86
-
GNU Parted (partition manager, create, destroy, resize, etc.)
- Tovid
(Burn video DVDs);
tovidwiz
(GUI w/ option to burn in subtitles);
unofficial Tovid help web site;
other extensions/alternatives for DVD are
cdrecord and cdrecord-ProDVD (CD/DVD burning),
OSS DVD (extension),
DVD+RW-tools,
cdrdao),
cdwrite mailing list)
-
freshmeat.net top 20 projects
(and top
20 projects by user rating)
- GNU ghostview, etc.
- GNU pdf reader/editor
(not much progress from 2011 - 2013; will allow pdf annotation)
- mutt (mail user agent)
(see also mutt intro
with pointers to other mail utilities)
- ImageMagick
(image manipulations with command, convert)
-
picresize.com (crop and compress images online; At the "crop"
page, be sure to use the mouse to select crop region before
selecting the "select crop" button.)
-
smallpdf.com (reduce size of pdf's -- but only two pdf's per hour
-- maybe by switching scanned PDF files to 144 dpi?)
- sam2p
(Creates small eps and pdf files from png, etc.,
by putting a wrapper around bitmap)
-
A png to eps converter (shell script using standard binaries)
-
PDF to PNG – Convert PDF to PNG Online
-
jpeg2ps
-
bmeps (to convert png, jpeg, netpbm to eps)
- gimp, xpaint for editing images (png is UNIX default bitmap -- includes ppm and others)
- display (from ImageMagick suite) or xv or xli for viewing images;
ImageMagick includes convert, import (screen capture),
mogrify, animate, montage, combine, xtp
- gimp (Photoshop equivalent)
(cf. Gimp tutorials)
- see also xfig for diagrams (or else "dia" for engineering circuits),
idraw, and graphviz elsewhere on this page
- Nvu (free, like
DreamWeaver and FrontPage, authoring web pages, etc.)
- Web Browsers: firefox (with tab-based browsing)
- Firefox uses ridiculous amounts of RAM (1/4 gigabyte of RAM for
"Hello, World" and almost no tabs or extensions;
this makes Firefox and other programs very slow. To help control this,
I set
browser.cache.memory.enable;false in
about:config. I've seen someone claim that
there is a browser.cache.memory.capacity
field for less drastic values, but I didn't see it in Linux.
- Ghostery Enterprise
(Cookie Consent Tool)
-
Lightweight Browsers (green browsers)
(w/ support for Flash)
- K-Meleon (Windows only)
TenFourFox (Firefox 4.x and higher for MacOSX 10.4 --
Choose G3 download for Intel)
Gnome/KDE shortcuts
Ubuntu Unity 11.10 keyboard shortcuts
Keynav (package for mouse move and click via keyboard; Debian pkg)
Pentadactyl
(also available from Firefox add-ons menu; keyboard-driven Firefox;
vim/links shortcuts)
Mouseless Browsing (MLB) (Firefox addon)
Window Managers with mouseless operation
- WMII (Window Manager Improved 2)
(small, dynamic window manager for X11; "It aims to maintain a small
and clean (read hackable and beautiful) codebase."; operates
independently of desktop environment)
- LXDE (lightweight desktop
environment (not just window manager); full-featured, limited
mouseless operation using
Openbox and editing ~/.config/openbox/lxde-rc.xml
to alter
default Openbox key bindings)
-
Midori (lightweight tab-based browser that can use some Firefox
extensions and plugins, like Flash)
- StumpWM
(tiling window manager based on Common Lisp)
- OpenBox (configurable window
manager that can operate inside Gnome or KDE)
- dwm (dynamic window manager)
sox (SOund eXchanger, with command, sox)
and lame (Lame Ain't an MP3 Encoder --- but it is :-) )
vsound
or alt
(also in Debian; records sound played through /dev/dsp (OSS);
not to be confused with VSound=Virtual Sound)
realrekord
(KDE graphic front-end to vsound)
How to Recording Internal Audio in Ubuntu
Amarok (good replacement for iTunes
jukebox)
Gtkpod (GUI for Apple Ipod)
PyMusique
(or alt)
- On a related area, see this summary of
DRM News
URL Clean (clean up URLs from Google search)
(demangles Google URL)
mplayer (and several others) for video
YT5s.com Youtube Downloader (be careful about dangerous links)
Online MP3 audio cutter
(or a video cutter)
Check out videolan.org (VLC)
Cloud Convrter (voic-to-text) (convert mps3,youtube video, etc., into test;
OCR (image-to-text) also available; approximate text, but refers
back to minute/second location of audio)
[Apparently only converts first 5 minutes to text.]
Cloud convert ("convert anythign to anything")
ffmpeg (documentation)
- NOTE: ffmpeg is now deprecated in favor of avconv
- 19 ffmpeg commands for all needs
-
HOWTO convert video files
- Compress video: ffmpeg -i <inputfilename> -s 640x480 -b 512k
-vcodec mpeg1video -acodec copy output.avi (Use size of 640x480,
bit rate of 512 kb/s, MPEG 1 video codec, and copy original audio),
all to avi (or choose .mpg) (-b:v for video bitrate on newer ffmpeg?
h264 is a better video codec)
- h264enc package uses mencode for H264 encoding (mencoder has
options for iso image, hard subtitles, etc.)
IMAP vs. POP (IMAP more easily supports "disconnected"
operation: offline e-mail, but keep mail on server, not on client)
Fetchmail Home Page
HylaFAX
LibreOffice
(open source, compatible with Microsoft Word; derived
from Open Office, in turn derived from StarOffice 5.2)
Grammalecte (grammar checker for French)
LanguageTool Style qnd Grammar Check
(designed for checking technical English)
Spanish dictionaries
OpenOffice bibliographic project (not that active as of 2010??)
Zotero bibliography extension for OpenOffice
Zotero firefox extension
Impress (.ppt) templates
Reee Open Office Impress Templates
catdoc and xls2csv and catppt (read doc,docx,xls,xlsx,ppt,pptx
as text file)
StarOffice (avail. for Linux, like Microsoft office,
free for private use,
(alt.))
AbiWord
(another clone of Microsoft Word, only 5 MB at last look)
Enchant (spell-checker library able to use multiple backends,
multiple languages)
Antiword
(Reading Microsoft Word in UNIX, recommended)
(also see freshmeat
project)
wvWare: Breaking the Hegemony (Reading Microsoft Word in UNIX) ---
also see (on-line viewer)
The banshee package is a good choice for Apple iPod support ???
Word2TeX (and TeX2Word, but sells for money)
pdftk (merge/Split/Rotate PDF Documents, etc.)
poppler-tools (includes pdfmerge, pdfseparate)
Command line tool to crop PDF files
(pdfcrop: removing oversized margins, and then print two to a page:
pdfcrop --margins -7 FILE.pdf
=> FILE-crop.pdf)
ps2book
(For booklet printing: pdftops -level3 file.pdf
and, for example, ./ps2book nbsp;-p nbsp;letter nbsp;-m nbsp;a4 nbsp;-F0.8 nbsp;file.ps
to produce "file_book.ps" and in okular print dialog, choose
"options/duplex printing/short side".)
bookletimposer - print two to a sheet (booklet)
Corel PHOTO PAINT® 9
Istanbul - GNOME Live
(video capture to Ogg Theora format; uses GStreamer)
CamStudio (free software variation of Camtasia;
unfortunately, only works on Windows)
xvidcap (video screen capture)
LiVES (Linux Video Editing System)
recordMyDesktop (video screen capture)
Borland Kylix Open Edition (C++ Integrated Development)
Piazza (free software replacement
for Blackboard)
SourceForge: Project Info -- Gaim
(or GAIM (alt.))
(clone of AOL's AIM with support for
IRC, ICQ, MSN, Napster, etc.)
Ximian
(productivity suite of 80 Gnome appplications,
file manager, gimp, dia (diagram creation), gnumeric (spreadsheet),
Evolution (e-mail), etc.,
with
Bonobo component system (embedded documents launch with
required application))
Gnome Hacks
SSH -- Secure Shell
7-Zip
(up to twice the compression of gzip)
Multimedia Stuff for Linux
- XBMC (media player controller for TV from
non-profit XBMC consortium)
-
MPlayer (Plays most video formats, DVDs, etc.)
- LAME
(encoding engine to encode MP3)
-
mp3splt - mp3 and ogg splitter
-
Mp3Wrap Project Homepage
- mplayer
(movie player + many audio/video formats)
- VLC streaming media player (handles many formats; capable
of snapshots of videos; command-line (--start-time=, --stop-time=, to
create snapshots))
- avconv (open source
video transcoder with command line; see 'man avconv' : EXAMPLES;
avconv -i $dir/$file" -s 400x240 -c:v mpeg4 -b:v 128k
-c:a copy "$dest")
- Handbrake
(open source video transcoder; )
- Transcode
(convert video formats, import DVD on the fly; also works on MacOSX;
related to VLC?)
-
List of video editing software (Wikipedia)
(or
overview from 2007)
- S1mp3.org - Firmware replacement for s1 mp3 player - Tools
(and S1mp3.org)
- Grip
(Gnome-based CD Player and ripper)
-
DVD Technical Guide (Pioneer)
- DVDrip (in perl)
- DVDrip (different software?)
- K9Copy (compresses DVD stream
to fit on 4.7 Gb recordable DVD)
- Vamps
Shrink DVD for backing up (shrink video, drop some audio tracks)
- Avidemux
Video editing (w/ great docs)
- XFree 86
(X for Linux, video drivers, etc.;
also see Xorg)
- Sound devices -- how to detect them, etc.
- RealPlayer 10 for Linux
(or
RealPlayer 10 Gold from Helix, as of June, 2006)
-
Helix RealPlayer for UNIX (plays RealAudio,look lower on page)
(with forum)
-
version of Helix player from Realaudio (Linux/UNIX:
http://docs.real.com/docs/playerpatch/unix/rv9_libc6_i386_cs2.tgz)
GNOME
(GNU Windows for Linux with
Gnome Sitemap)
-
GNOME Office (productivity applications for GNOME
Desktop, including AbiWord (word processor), Gnumeric (spreadsheet),
GIMP (image editing), Eye of GNOME (image viewer),
GNOME-PIM (address/calendar/Pilot connectivity), GNOME-DB (database
connectivity) )
- GTK+ (GTK = GIMP Toolkit, GUI used
in GIMP and in GNOME, with bindings to many languages, and excellent
GTK tutorial and
GTK+ Reference Manual; GTK+ uses GDK and Glib (below))
- GDK
(X11-based widget library)
- Glib
portable C extensions for basic data types, data structures,
memory allocation, linked lists, trees, hash tables, arrays
that grow dynamically, relations
and tuples (databse), lexical scanner, string completion, IO channel
(file, pipe or socket), dynamically loading modules (wrapper for
dlopen(), etc.),
threads, main event loop, timers, etc.
(especially, portability between UNIX and Windows)
- GLADE GTK+ User Interface
Builder
-
Qt Designer (... is to KDE/Qt as Glade is to Gnome/GTK+)
-
GNOME Office - Dia (Windows Visio-like diagram drawing
application for GNOME office; but also see
Graphviz)
- GNOME Pilot
(Connectivity and sync with Palm Pilot)
- XML C library for Gnome
- XML in Gnome
-
Gnome Programming Guides, including:
-
Gnome Sitemap
-
Corba and the Gnome
- Gnome API's
- Gnome Tutorials
- Gnome Developer's FAQ
- Gnome Whitepapers
(including XML/DOM, Corba, Gnome Canvas, Gnome 1.0 Library Roadmap,
Drag-and-Drop, Esound (ESD), Multimedia, Components,
Gnome Configuration, etc.)
- Pointers to
other standards (DOM, CORBA, XML, etc.)
- i3wm: improved tiling wm
-
Xorg's X Window Innovation (latest info on X11/Linux, 5/30/09)
-
Free Graphics Tutorials
Learn OpenGL" (excellent!,
covers recent versions of OpenGL)
OpenGL 3.0 Tutorials not using any deprecated functionality
GLES tutorial
(relevant to quickly learning basics of OpenGL 1.5
with vertex arrays and textures)
OpenGL Tutorials (Lazy Foo' Productions; uses freeGLUT;
excellent starting point for older OpenGL 1.x standard)
OpenGL
-
ApiTrace (for tracing/replaying OpenGL and Direct3D calls)
- BuGLe (for tracing
OpenGL calls)
- VirtualGL.org
(GLX Forking, aka split rendering)
Simple DirectMedia Layer (SDL)
(typically used for creating a window, a graphics context or
using the mouse and other input devices)
GLUT-like Windowing Toolkits (GLUT was one of the first
cross-platform libraries for creating a graphics context for
OpenGL, etc., this is a list of other windowing toolkits,
such as SDL.)
WebGL (Wikipedia:
"JavaScript 3D graphics API based on OpenGL ES 2.0
using canvas element accessed with DOM")
Vulkan
The Gnome and Corba (book with full introduction)
Annotated Guide to XML
(by O'Reilly & Associates and Seybold Publications)
-
Extensible Markup Language (XML)
- XML 1.0 Specification
-
Explanation of XML
-
Annotated XML(requires Javascript) (or
XML
without Javascript)
- related XML technologies
- XML 1.0 spec
(eXtensible Markup Language, subset of SGML,
includes DTD (document type definition)
for verifying correct format of document)
- XML Protocol
(peer-to-peer communication via XML)
- XML Schema
(replacement for DTD, defining structure and content of XML document)
- XMLQuery
(extracting data from XML documents)
-
XML Linking (linking resources, e.g.: embedding remote documents,
includes XPath (addressing into XML tree), XPointer (reference fragment
of XML doc using XPath),
XML Base (compatibility with HTML), XLink (hot links for embedding))
- RDF
(Resource Description Format, e.g.: replacement for
META tag for HTML document keywords)
- DOM
(Document Object Model, standard for dynamic HTML)
-
HTML5 Tutorial (with HTML5 built-in elements for video, audio, etc.)
-
HTML5 differences from HTML4
- CSS
(Cascading Style Sheets)
- Scroll to Text Fragment
(using #:~:text=START,END)
-
JavaScript and HTML DOM Reference
- XSL
(language for expressing stylesheets)
- XSLT
(language for transforming XML documents, tree transformations)
- XHTML
(extension of HTML to standard including newer technologies)
-
S5: A Simple Standards-Based Slide Show System
(based on XHTML, CSS, and JavaScript)
- MathML
(Mathematical Markup Language, representing math expressions
in HTML)
- SMIL
(Synchronized Multimedia Integration Language,
multimedia onthe Web, e.g. Shockwave)
- SVG
(Scalable Vector Graphics, diagram drawing/plotting in XHTML)
- XML Signature and Canonicalization
(cryptographic signatures, author verification, etc.)
-
Simple Object Access Protocol (SOAP)
(XML-style marshaling of structured data,
à la XDR; also encapsulation of RPC call as remote method call)
- Firefox browser
- Javascript FAQ
- NodeJS (Javascript-like scalable
network apps, such as tiny web server, concurrency with single thread;
uses Google's open source
V8 JavaScript engine)
-
C Socket Programming for Linux with a Server and Client Example Code
-
TCP Server-Client implementation in C - GeeksforGeeks
- Oleo spreadsheet
- AFS
Productivity Stuff for Windows
-
Emacs (and pointers to many other UNIX utilities)
- GSView
(view postscript and pdf on Windows -- also needs
ghostscript)
-
Ghostscript home page
-
GhostScript-to-PDF converter (ps2pdf, note also pdflatex and pdftex
from TeTeX; IMPORTANT: Use "dvips -P cmz -t letter" to avoid
fuzzy fonts in pdf)
- dvipdfm
(dvi to pdf translator)
-
Tips on ssh for windows (including Teraterm Pro/ttssh - a better
terminal emulator)
-
MikTeX (Others , e.g. tetex, tex-k, also exist)
-
Tiny Tex FAQ (based on Tex Live; See especially:
6. "... How can my users without root privileges install LaTeX packages by themselves?")
- Tiny Personal Firewall
(I know. Not UNIX stuff, but I didn't have another section for it.)
- Win32 viewer, et al.
(can create connection to UNIX X-windows session from Windows, and
with optional
port forwarding through an ssh client)
- FreeZip
(free unzipping, ~ 250KB)
- Midnight Commander
(powerful file system browser for Windows, UNIX/Linux, etc.)
-
ISO Recorder (for burning images; also
Padus DiscJuggler,
free trial limited to 1x Burn Speed)
-
Security and other useful Windows software
(free and/or cheap; protect against viruses, spyware, trojan horses,
popup windows, and other features of the Windows world -- in French,
but well worth reading)
Java, etc.
HTML, etc.
Distributed Computing
PC/computer and trade magazines
Other PC-related pages
- Tom's Hardware Guide
- Silent PC Review
- CTS Computers
(local to Boston)
- Crucial.com
(offers memory selector to match RAM type to motherboard)
New Technologies
Other Stuff
- The Network Simulator - NS-2
- Cube Lovers' archive
-
IRC: A Short Primer
- Wavelets
-
Computer Science Resources
-
Job outlook, C.S. (Bureau of Labor Statistics)
- Winsock shareware
- Jumbo! Shareware Archive
- Virtual Shareware Library
- Macfixit (helps fix Mac problems)
- AGNetTools for network trace/ping/etc. (Mac and Windows)
- Emacs Lisp Introduction
-
Common Lisp Repository
-
The Association of Lisp Users -- Lisp Archives Page
-
The Internet Scheme Repository (U. Indiana)
-
Artificial Intelligence
-
Natural Language Processing
- CleverBot (similar to Eliza, but
trained on millions of conversations)
- Microsoft
- Lynx Home Page
- Walnut Creek software archives
(also available on CD)
- Download.com (software archive)
- (Also try xnetlib)
-
Offline NT Password & Registry Editor
-
Beginners Guides: Forgotten Passwords & Recovery Methods
- LaTeX2e unofficial reference manual
- Typist (a nice replacement for LaTeX)
-
Hypertext Help with LaTeX
- TeX (Wikibooks)
- LaTeX (Wikibooks)
- LaTeX/Package Reference (Wikibooks: common LaTeX packages)
- Conference Posters in Linux
(poster preparation via LaTeX)
- The Comprehensive LaTeX Symbol List
- Embedding Times font in LaTeX pdf file:
pdftops file.pdf
ps2pdf14 -dPDFSETTINGS=/prepress file.ps
-
overfull lines (fixing bad line breaks, e.g., in two-column mode)
(excellent!)
- LaTeX/page layout (\topmargin, \textheight, etc.)
- Packages in the `graphics' bundle (for LaTeX)
- Convert PDF to PDF/A (original):
gs -dPDFA=1 -dNOOUTERSAVE -sProcessColorModel=DeviceRGB -sDEVICE=pdfwrite -dPDFACompatibilityPolicy=1 -o output.pdf input.pdf
- OR: convert PDF to PDF/A (better):
gs -dPDFA -dBATCH -dNOPAUSE -sColorConversionStrategy=UseDeviceIndependentColor -sDEVICE=pdfwrite -dPDFACompatibilityPolicy=2 -sOutputFile=output.pdf input.pdf
(-dPDFACompatibilityPolicy=2 includes DOCINFO,
but version 1 also works; see here)
- Merge, convert, and compress files
and e-mails to PDF and PDF/A
(also try: \documentclass[pdfa]{...} ; \usepackage[1-b]{pdfx} ;
for PDF/A-1b)
- verapdf.org (free downloadable
PDF/A verification tool)
- PDF/A validation tool
(online)
- Overleaf.com (like Google Docs, but based on LaTeX; LaTeX source can then be downloaded and customized locally)
- Automatically generate LaTeX tables
-
LaTeX for Complete Novices (excellent introduction:
complete book)
-
Excellent introduction to LaTeX
-
A Guide to LaTeX (excellent for beginners)
- Inessential LaTeX
- LaTeX-Befehlsreferenz
-
The Comprehensive LaTeX Symbol List (.pdf)
-
Embedding graphics in LaTeX (or
in Chinese)
- TeX (concise, comprehensive
help on LaTeX,BibTeX, MakeIndex, SliTeX, etc.)
-
Tricks for latex graphics and figures (dynamic conversion
of graphics, etc.)
-
Macros: a complement to \smash, \llap, and \rlap
(and to \phantom, \vphantom, \hphantom; by Alexander R. Perlis)
-
LaTeX Packages
- Help on LaTeX Commands
-
LaTex Tips and Tricks (Advanced LaTeX, excellent)
-
LaTeX tricks (Advanced LaTeX, quite good)
-
References and tutorials for LaTeX
-
texlive install (NOTE: "By the way, native TeX Live, which
is typically installed under /usr/local, can happily coexist with
a TeX from your operating system, each with their own completely
independent trees and programs. (Do not try to merge them!) So you
can install native TeX Live if your vendor is not keeping up.")
-
Tiny TeX: A ligtheweight and easy-to-maintain LaTeX distribution
(based on Tex Live; Note
tlmgr --usermode install PKGs;
Note updmap-user)
-
TeX Frequently Asked Questions
- ctan.org (Comprehensive
TeX Archive Network)
-
teTeX Home Page
-
writeLaTeX (multiple interfaces to document on web,
calls latex on it automatically)
-
TeX archive
-
TeX archive
-
New LaTeX
- The essential guide to LaTeX
(introduction to LaTeX)
- The (not so) short guide to LaTeX 2e
- LaTeX manual
- The LaTeX survival guide (good on figures)
-
Catalogue of LaTeX packages
-
The TeX System (index of further information)
-
LaTeX3 project
-
Online Computer Manuals
- SPEC benchmarks
-
IA-64 architecture (Intel Merced, w/ nice
overview and
tutorials)
-
IA-64 architecture (Hewlett-Packard)
-
Digital HPC
- Sun Microsystems
-
SPARC CPU's
- Microsoft
- Intel
-
Intel Pentium 4 Processor Home Page
(w/ manuals, data sheets, papers, applic. notes, etc.)
- x86-64.org
- ARMv8 Instruction Set Overview
(64-bit ARM)
-
A Gentle Introduction to Assembly Language Programming
-
The RISC-V Instruction Set Manual, Volume I: User-Level ISA,
Version 2.0
- AMD technical documentation
- Weak vs. strong memory models (and sequential consistency)
- Multiprocessor Systems
- CMU C.S. projects
- Apple developer site
-
The Info-Mac HyperArchive
-
Macintosh Software
-
MacTCP Unleashed!
- DIMACS archive
- Knuth-Bendix
(Charles Sims)
-
authentication
Professional Societies
Some German societies
Conference Calls
gotomeeting.com
(with screen-sharing by organizer w/ their desktop app;
30-day free trial available; free web interface for non-organizers)
Calendars of Upcoming Conferences
Funding agencies
Math, etc.
Standards
Links to other documentation
Request For Comments (RFC) Home
RFC (Request for Comments)
Cygnus library of free (GNU-related) documentation
Modem info (FAQ, etc.)
Overview of HTTP
IETF - Hypertext Transfer Protocol (HTTP) Working Group
SPEC benchmarks
Geekbench 5 benchmarks (for Macs)
Intellectual property (includes
software patents, copyrights, etc.)
