Internet / WWW

CMS (content management)

Correu electrònic / E-mail

Correu brossa / Spam

Dominis / Domains

Països / Countries (ISO 3166)
- ISO country codes (RIPE ftp)
- ISO country codes
- Country codes (Proxys 4 all)
Distribution by Top-Level Domain Name by Name
Arsys
iDirections
Interdomain
Nominalia
Whois (domaintools)
Nic.es
dot.tk (free domains for all)

Allotjament / Hosting

	apps	manage	url	capacitat	baixada	comments
Google Drive	-		https://googledrive.com/host/0B42MQgikjmMwT05OV09ONDR4WkE/			Is it possible to have clean URLs for Google Drive items? [closed]
	gdriv.es		http://gdriv.es/francescpinyol/			only highest level url
	gweb.io					$1.50/month
Dropbox	-		https://dl.dropboxusercontent.com/u/8062953/site/index.html	(Dropbox)
	dockbox.io					dockbox.io – Host Websites in Dropbox With Custom Domains
	harp.io					Harp.io – Host production websites with custom domains and SSL
	KISSr	http://www.kissr.com/sites	http://francescpinyol.kissr.com/	50MB	50MB/mes
	DropPages					based on given templates
	Site44				100MB/mes
allotjament.cat
webgratis.cat			http://francescpinyol.webgratis.cat/	1GB	100GB/mes	PHP, MySQL

	alta domini	renovació (¤/any)		emmagatzematge	tràfic	correu		altres			preus
						comptes	espai	MySQL	PHP	One Click	mensual	anual
active24	8	16,99 8	Basic Hosting	5GB	il·limitat			x	x	x	4,99 1,99
swhosting	8	17,75	Petit L	2GB	5GB/mes			-	x	-	3,50
Don Dominio (.cat)	9,95	22,95 17,95	Pla Mini	100MB	5GB/mes	5	50MB	-	-	-	0,08	1
Don Dominio (.cat)	9,95	22,95 17,95	Pla Basic	1GB	12GB/mes	100	300MB	100MB	x		2,08	25
Dina Hosting	8,95	18,60	Hosting Personal	2GB	30GB/mes	20					4,50
Virtual Pyme	8,95	18,95	Mínim	1GB	5GB/mes	10					2,45
CD Mon	8	19,95	Pla Start	1GB	10GB/mes	1		-	x		2,50	30
Entorno Digital	8	19,99	Pla Presència	2GB	10GB/mes	5		-	-		3,80

Feeds

RSS
- Building and Using an RSS Feed

Geolocalització / Geolocation

GPS
Geolocation (wp)
Geolocation software (wp)
HostIP (community-based)
MaxMind (commercial / free)
IP geolocation (The NetOp Organization)
Google Gears
- API Blog
- Google Gears Geolocation API, example displays your location on Google Map
Firefox
- Geolocation (add-on)
- Using geolocation (Firefox 3.5)
What is my geolocation
W3C Geolocation API (wp)
Geolocation in Firefox and Beyond
Where 2.0 - Location on the Web
- Search Flickr Photos based on Time and Geolocation

Grups de notícies / Newsgroups

Gestors de preferits / Bookmarks managers

Bookmarks managers on Google

Internet per satèl·lit / Internet via satellite

Internet Services Providers (ISP)

Internet TV

Freedomland
Ultimate TV (també/also DVR)
Projectes / Projects
- ClicaTV (Generalitat de Catalunya)

Internet of things

Open Smart Cities I: Open Internet of Things
Maquinari / Hardware
- Sistemes / Systems

Protocols

TCP and UDP port numbers (Wikipedia)
Internet Protocol (IP)
- Geo IP C API
- Get my IP
  - STUN
  - checkip.amazonaws.com
  - WhatsMyIP.org
  - HOWTO: Check you external IP Address from the command line
    - wget -q -O - checkip.dyndns.org | sed -e 's/.*Current IP Address: //' -e 's/<.*$//'
  - WhatIsMyIP
- Follow Me IP
- DNS providers
  - List of managed DNS providers (wp)
  - AWS Route53
  - DynDNS
    - DynDNS update clients
    - ddclient

W3C

HTTP

HTTPS

Securing the web
Servidors / Servers
- Cipherli.st Strong Ciphers for Apache, nginx and Lighttpd
  - SSL Decoder (test your server)
  - Cipherli.st: Configuraciones SSL robustas para servicios
- SSL Server Test (Qualys SSL Labs)
- Problems
  - Client (browser) warnings about mixed content
    - The Protocol-relative URL
    - Solució / Solution
      - Use network-path reference (RFC 3986)
- Certificats / Certificates
  - OpenSSL
    - Certificats autosignats / Selfsigned certificates
  - Let's Encrypt
- Apache server with HTTPS
- Nginx with HTTPS
Clients
Server Name Indicator (SNI) (wp)
- from nginx docs:
  - "With this configuration a browser receives the default server’s certificate, i.e. www.example.com regardless of the requested server name. This is caused by SSL protocol behaviour. The SSL connection is established before the browser sends an HTTP request and nginx does not know the name of the requested server. Therefore, it may only offer the default server’s certificate."
  - "A more generic solution for running several HTTPS servers on a single IP address is TLS Server Name Indication extension (SNI, RFC 6066), which allows a browser to pass a requested server name during the SSL handshake and, therefore, the server will know which certificate it should use for the connection."
- SNI in nginx
HSTS
- HSTS on nginx
Marca blanca / White label
- Info
  - Letsencrypt
    - How Can I Get A Certificate Not For My Server?
    - How to generate the certificates more programatically
      - peter_sslers
        
        Pyramid
        
        small, fast, down-to-earth, open source Python web framework
        
        Django vs Flask vs Pyramid: Choosing a Python Web Framework
    - Automate SSL cert generation and renewal using REST API
      - CNAME to _acme_challenge
      - acme-dns
    - certbot-s3front plugin
  - How to White-label a Single-Page App
  - How to handle SSL certificates for implementing WhiteLabel option in a web app running on NGINX server
  - AWS
    - Can we build a domain Whitelabel multi-tenancy system with S3 and cloudfront?

Let's Encrypt

wildcards
Protocols
- ACME (Automatic Certificate Management Environment)
  - RFC 8555
    - 8. Identifier Validation Challenges
  - How it works

Documentation

Challenge types

ACME challenge	used by	pros
	certbot plugin
HTTP-01	apache nginx webroot standalone manual	It allows hosting providers to issue certificates for domains CNAMEd to them. ...
DNS-01	DNS plugins ... manual

ACME Clients

certbot

Documentation
Installation
- Running the client with Docker
- Mageia 9
  - sudo dnf install certbot
- CentOS / Alma
  - yum install certbot python2-certbot-apache bind-utils
Compilation
- Requirements
  - CentOS
    - sudo yum install python2-mock python-zope-interfacepython-zope-component python-six python-setuptools python2-acme
  - Mageia
    - Compile certbot (recommended)
    - or get it:
      - wget https://dl.eff.org/certbot-auto
      - chmod a+x certbot-auto
      - ./certbot-auto -debug
      - ./certbot-auto certonly
- Download
  - git clone https://github.com/certbot/certbot.git
- Compile
  - cd certbot
  - sudo python setup.py install
- Compile version >=0.22 (for wildcards)
  - cd certbot
  - virtualenv env
  - source env/bin/activate
  - pip install --upgrade pip
  - pip install --upgrade setuptools
  - cd acme
  - python setup.py install
  - cd ..
  - python setup.py install
  - sudo ./env/bin/certbot ...
Configuració / Setup
- serveis amb usuari no root / non-root services
  - How to use certs in non-root services?
  - letsencrypt_change_permissions.sh
    - #!/bin/bash # file permissions chgrp ssl-cert -R /etc/letsencrypt chmod 2755 /etc/letsencrypt chmod g=rX -R /etc/letsencrypt
  - groupadd --gid 3000 ssl-certchange_letsencrypt_persmissions.sh
    usermod -a -G ssl-cert nginx
  - crontab
    - certbot -n renew --deploy-hook /usr/local/bin/letsencrypt_change_permissions.sh

Usage

	info	usage from		additional setup for running server
		CLI options	/etc/letsencrypt/renewal/your.domain.org.conf	Apache	Nginx
		`certbot` ...	`[renewalparams]`	/etc/httpd/httpd.conf	/etc/nginx/nginx.conf
authentication	when your html pages are on a static server (hosted, AWS S3...)	`--authenticator manual`
	when you have no running server, but ports 80 and 443 are available. A mini server will be started just for the process. E.g.: Postfix email server	`--authenticator standalone`	`authenticator = standalone`
	when you have a running server (certbot will generate some content locally, in /path/to/.well-known/, which must be available by http from letsencrypt servers; this requires additional setup of your server) Can also be used to obtain a certificate for Postfix (`-d mail.domain.org`) when an http server is already running (even if it is running with `server_name your.domain.org`). But make sure that you have a DNS entry: `mail.domain.org CNAME your.domain.org`	`--authenticator webroot -w /path/to`	`authenticator = webroot [[webroot_map]] your.domain.org = /path/to`	(needed?) `<VirtualHost *:80> Alias /.well-known /path/to/.well-known <Directory "/path/to/.well-known"> Options Indexes FollowSymLinks MultiViews AllowOverride All Require all granted </Directory>`	`server { listen 80; server_name your.domain.org; # letsencrypt location /.well-known { alias /path/to/.well-known; }`
	when you have a running Apache server	`--authenticator apache`	`authenticator = apache`
	when you have a running Nginx server	`--authenticator nginx`	`authenticator = nginx`

installation	you do not want to install the obtained certificate	`certonly`	`installer = None`
	you want to install the obtained certificate in the Apache config	`--installer apache`	`installer = apache`
	you want to install the obtained certificate in the Nginx config	`--installer nginx`	`installer = nginx`

Examples
- Just retrieve a certificate (a nginx server is running):
  - webroot=/usr/share/nginx/html letsencrypt_port=80 key_size=4096 email=me@my_company.com domain=my_subdomain.my_company.com sudo certbot certonly -n --agree-tos --webroot -w $webroot --http-01-port $letsencrypt_port --rsa-key-size $key_size --email $email -d $domain
- just retrieve a wildcard certificate (requires certbot version >=0.22)
  - install the required dns plugin
    - AWS Route 53
      - AWS Setup
        
        Role of instance (or user) must have the certbot-dns-route53 sample policy
      - sudo yum install certbot-dns-route53
    - ...
  - sudo -i letsencrypt_port=80 key_size=4096 email=me@my_company.com domain='*.my_company.com' certbot certonly -n --agree-tos --dns-route53 --http-01-port $letsencrypt_port --rsa-key-size $key_size --email $email -d $domain--server https://acme-v02.api.letsencrypt.org/directory
  - Problemes / Problems
    - The currently selected ACME CA endpoint does not support issuing wildcard certificates.
      - Solució / Solution
        
        --server https://acme-v02.api.letsencrypt.org/directory
- just retrieve a certificate using AWS Route 53
  - install the required dns plugin
  - sudo -iemail=me@my_company.com domain='mydomain.my_company.com'
    certbot certonly [--dry-run] -n --agree-tos --dns-route53 --key-type ecdsa--email $email -d $domain
Renewal
- Renewing certificates
  - --pre-hook
  - --post-hook
  - --deploy-hook
- all certificates:
  - certbot -n renew
- specified certificates:
  - certbot certonly -n --authenticator webroot --webroot-path /path/to -d your.domain.org

Client options

Apache
- first time
  - certbot run --apache --http-01-port 80 --rsa-key-size 4096 --emailyour@email.org -d your.domain.org
  - non interactive:
    - certbot run --apache -n --agree-tos--http-01-port 80 --rsa-key-size 4096 --emailyour@email.org -d your.domain.org
- first renewal (because apache authenticator is not working for certain apache config)
  - certbot certonly -n --authenticator webroot --webroot-path /path/to/ -d your.domain.org
- next renewals
  - sudo certbot -n renew
  - Problemes / Problems
    - /etc/letsencrypt/options-ssl-apache.conf not found
      - Solució / Solution
        
        sudo yum install python2-certbot-apache
        
        ln -s /usr/lib/python2.7/site-packages/certbot_apache/options-ssl-apache.conf /etc/letsencrypt/options-ssl-apache.conf
    - Could not open configuration file /etc/letsencrypt/options-ssl-apache.conf: Permission denied
      - This happens because /etc/letsencrypt is a symbolic link to /mnt/nfs/letsencrypt, a nfs-mounted point
      - Solució / Solution
        
        SELinux in HTTP servers
        
        setsebool httpd_use_nfs 1
Nginx (using webroot)
- first time
  - certbotcertonly --webroot -w /path/to --http-01-port 80 --rsa-key-size 4096 --email your@email.org -d your.domain.org
  - non interactive:
    - certbotcertonly -n --agree-tos --webroot -w /path/to --http-01-port 80--rsa-key-size 4096 --email your@email.org -d your.domain.org
- renewal
  - /etc/nginx/conf.d/my_site.conf
    - server { # letsencrypt location /.well-known { alias /path/to/.well-known; } ... }
  - check that /etc/letsencrypt/renewal/my.site.org.conf contains:
    - ... [renewalparams] authenticator = webroot webroot_path = /path/to installer = None ...
  - sudo certbot renew --post-hook "systemctl restart nginx.service"
  - crontab
    - certbot -q -n renew --post-hook "systemctl restart nginx.service"
Certificate installation in hosting services
1. Get only the certificate (and follow the instructions)
  - domain=my_subdomain.my_company.org sudo certbot certonly --authenticator manual -d $domain
2. Install it:
AWS
- if you want to use certificates for LoadBalancer or CloudFront, consider using AWS ACM instead
- EC2
  - Adding A “Let’s Encrypt” SSL Certificate to an Amazon AWS Instance
- Elastic Load Balancer (ELB)
  - Let's encrypt and ELBs?
  - First time
    1. create a load balancer with a port 80 listener
    2. AWS IAM: grant permissions to user/role that will be used in next steps
      - elasticloadbalancing:CreateLoadBalancerListeners
      - iam:GetServerCertificate
      - iam:UploadServerCertificate
      - Example of policy:
        
        { "Version": "2012-10-17", "Statement": [ { "Sid": "", "Effect": "Allow", "Action": [ "elasticloadbalancing:DescribeLoadBalancers", "elasticloadbalancing:CreateLoadBalancerListeners", "elasticloadbalancing:SetLoadBalancerListenerSSLCertificate" ], "Resource": [ "*" ] }, { "Sid": "", "Effect": "Allow", "Action": [ "iam:ListServerCertificates", "iam:GetServerCertificate", "iam:UploadServerCertificate" ], "Resource": [ "*" ] } ] }
    3. from an instance (e.g. from an instance behind the load balancer)
      1. get a LE certificate (IMPORTANT: keys with 4096 bytes are not allowed; only 2048 or 1024 (not allowed by Let's Encrypt))
        
        certbot ... --rsa-key-size 2048 ...
      2. upload the certificate to IAM (Upload the Certificate):
        
        aws iam upload-server-certificate --server-certificate-name ${domain}.cert \ --certificate-body file:///etc/letsencrypt/live/${domain}/cert.pem \ --private-key file:///etc/letsencrypt/live/${domain}/privkey.pem \ --certificate-chain file:///etc/letsencrypt/live/${domain}/chain.pem
      3. Get the ARN of the uploaded certificate
        
        ARN=$(aws --output json iam get-server-certificate --server-certificate-name ${domain}.cert | jq '.ServerCertificate.ServerCertificateMetadata.Arn')
      4. Add an HTTPS Listener Using the AWS CLI
        
        aws elb create-load-balancer-listeners --load-balancer-name my-load-balancer --listeners Protocol=HTTPS,LoadBalancerPort=443,InstanceProtocol=HTTPS,InstancePort=443,SSLCertificateId=$ARN
      5. ...
  - Renewal
    - letsencrypt-aws
      - Installation
        
        git clone https://github.com/alex/letsencrypt-aws.git
        
        cd letsencrypt-aws
        
        virtualenv env
        
        source env/bin/activate
        
        pip install -r requirements.txt
        
        deactivate
      - First time
        
        ask ACME for a private key
        
        source env/bin/activate
        
        python letsencrypt-aws.py register email@host.com
        
        deactivate
        
        save it as email_host_com.privkey.acme.pem (locally or on AWS S3)
        
        create an IAM policy (name it e.g. LetsencryptAWS)
Problemes / Problems
- The requested nginx plugin does not appear to be installed? #1736
- Solution: manual request and installation
  - cd letsencrypt
  - ./letsencrypt-auto certonly -a manual --rsa-key-size 4096 --email your@email.org -d your.domain.org
  - /etc/nginx/nginx.conf (or /etc/nginx/conf.d/your_site.conf)
    - ssl_certificate /etc/letsencrypt/live/www.example.org/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/www.example.org/privkey.pem;
  - Note: if you specify cert.pem instead of fullchain.pem, browsers will work, but curl nor openssl s_client won't:
    - curl will report:
      - curl: (60) SSL certificate problem: unable to get local issuer certificate More details here: https://curl.haxx.se/docs/sslcerts.html curl failed to verify the legitimacy of the server and therefore could not establish a secure connection to it. To learn more about this situation and how to fix it, please visit the web page mentioned above.
    - openssl s_client will report:
      - CONNECTED(00000003) depth=0 CN = ... verify error:num=20:unable to get local issuer certificate verify return:1 depth=0 CN = ... verify error:num=21:unable to verify the first certificate verify return:1
    - Configuring nginx for letsencrypt
    - SSL certificate chains (nginx.org)
    - Let's Encrypt CA not included in Ubuntu's CA bundle #2026
    - The requested nginx plugin does not appear to be installed? #1736

Content negotiation
- Single URI for several files (URLs)
- Content Negotiation: why it is useful, and how to make it work
- Fragment identifiers and content negotiation (Architecture of the World Wide Web, Volume One)
- HTTP request headers
  - Format: Accept
  - Language (i18n): Accept-Language
  - Encoding: Accept-Encoding
- i18n
JPEG JFIF
SOAP (Simple Object Access Protocol)
Platform for Privacy Preferences (P3P) (press release)
Resource Description Framework (wp) (metadades/metadata)
Media Fragments URI (MFWG)
- <scheme name>:<hierarchical part>[?<query>][#<fragment>]
- W3C Media Fragments (NinSuna)
  - NinSuna illustrating W3C Media Fragments delivery
  - NinSuna (The Handbook of MPEG Applications)
- Implementing the Media Fragments URI Specification (pdf)
- HTTP streaming using media fragments
WebVTT (Web Video Text Tracks)
Cross-Origin Resource Sharing (CORS)
- Access-Control-Allow-Origin: http://foo.example/
W3C TV

IETF RFC

1630: URL
3986: URL
- Appendix B. Parsing a URI reference with a regular expression
- Percent-encoding (URL encoding) (wp)
  - HTML URL Encoding Reference (w3schools)
  - curl
    - --data--urlencode
  - Conversion in Python:
    - URL Quoting
1867: File upload
2076: Common Internet Message Headers
2326: Real Time Streaming Protocol (RTSP)
2445: Internet Calendaring and Scheduling Core Object Specification (iCalendar)
2616: HTTP 1.1
- Content-Range header (used by progressive download)
- Content-Length
- 2.2 Basic Rules
  - CR LF
- 3.6.1 Chunked Transfer Coding (wp)
  - Example
- 6.1.1 Status Code and Reason Phrase
2617: HTTP Authentication
2965: Cookies
3875: CGI
3926: FLUTE File Delivery over Unidirectional Transport
- Implementations
  - Compilation
  - MAD FLUTE (Tampere)
  - DelCo (Tampere)
  - FLUTE/ALC and FCAST/ALC (MCLv3) (Inria)
    - Download
    - JFlute
- Citation from other standards
  - DVB-IPDC
    - TS 102 472: CDS
    - FLUTE DVB-H
  - OIPF
    - Volume 6: PAE
7231: Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content

JSON Web Token (JWT) (RFC 7519) (wp)

<base64url(header)>.<base64url(payload)>.<base64url( sign( <base64url(header)>.<base64(payload)> ) )>

parts

part group fields

header

commonly-used

`typ`	token type	`JWT` `...`
`cty`	content type
`alg`	algorithm	`HS256` `RS256` `...`
`kid`	key id
`x5c`	X.509 certificate chain
`x5u`	X.509 vertificate chain url
`crit`	critical

payload

registered claims

`iss`	issuer
`sub`	subject
`aud`	audience
`exp`	expiration date
`nbf`	not before
`iat`	issued at (epoch)
`jti`	JWT ID

public claims

private claims

signature (hash + xifratge)

Implementation requirements (RFC7519 section 8) (JWA)

signature		`alg`
secret (symmetric)	HMAC using: SHA-256 SHA-384 SHA-512	HS256 HS384 HS512
public/private key pair (asymmetric)	RSASSA-PKCS1-v1_5 using: SHA-256 SHA-384 SHA-512	RS256 RS384 RS512
	ECDSA using: P-256 and SHA-256 P-384 and SHA-384 P-521 and SHA-512	ES256 ES384 ES512

encryption (usually not needed) (JWA)


MUST	RSAES-PKCS1-v1_5 with 2048-bit	RSA1_5
	AES Key Wrap with 128- and 256-bit keys	A128KW A256KW
	AES-CBC and HMAC SHA-2	A128CBC-HS256 A256CBC-HS512
RECOMMENDED	ECDH-ES	ECDH-ES+A128KW ECDH-ES+A256KW
	GCM	A128GCM A256GCM

JSON Web Algorithms (JWA) (RFC7518)
Auth with JSON Web Tokens
Cookies vs Tokens. Getting auth right with Angular.JS
JWT.io (interactive validation)
- Blacklisting JSON Web Token API Keys
Ús / Usage
- Amazon CloudFront
  - amazon-cloudfront-functions/kvs-jwt-verify

Implementacions / Implementations

Python

JWT in Django DRF

PyJWT

Installation
- pip install PyJWT

Usage examples

	HS256	RS256
	import jwt
create	encoded = jwt.encode({"some": "payload"}, "secret", algorithm="HS256")
verify

bash
- How do you create an RS256 JWT assertion with bash/shell scripting?
- Generate JWT With Bash
- generate-jwt.sh (github: amazon-cloudfront-functions/kvs-jwt-verify)
- Create token
  - #!/bin/bash # header header='{"alg":"RS256","typ":"JWT"}' header_b64=$(echo -n $header | base64 | tr -d '\n' | tr -d '=' | tr '/+' '_-') # claim set / payload claim_set='{"sub": "1234567890","name": "John Doe", "admin": true}' claim_set_b64=$(echo -n $claim_set | base64 | tr -d '\n' | tr -d '=' | tr '/+' '_-') # signature of header concatenated with claim_set (in base64): by definition, is encryption (with private key) of hash header_claim_set_b64=${header_b64}.${claim_set_b64} #signature=$(echo -n "$header_claim_set_b64" | openssl sha -sha256 -sign private.key | base64 | tr -d '\n' | tr -d '=' | tr '/+' '_-')signature=$(echo -n "$header_claim_set_b64" | openssl dgst -sha256 -sign private.key | base64 | tr -d '\n' | tr -d '=' | tr '/+' '_-') # all jwt=${header_claim_set_b64}.${signature}
- Verify token
  - ...
- ...
Javascript
- verify-jwt.js (github: amazon-cloudfront-functions/kvs-jwt-verify)
...

		base64
header	{"alg":"RS256","typ":"JWT"}
claim set
signature

Protocols de missatgeria / Messaging protocols

	broker implementation (MOM)	clients
		Android
AMQP	RabbitMQ
MQTT	Mosquitto	MQTT Dash (IoT, Smart Home)
STOMP	ActiveMQ RabbitMQ

AMQP (Advanced Message Queuing Protocol) (wp)
- Understanding AMQP, the protocol used by RabbitMQ
- RabbitMQ
MQTT (Message Queue Telemetry Transport) (wp)
- Brokers
  - mqtt (Mosquitto)
    - Install
      - sudo urpmi mosquitto
    - Start
      - systemctl start mosquitto.service
- Clients
  - Android
    - MQTT Dash (IoT, Smart Home)
  - CLI
    - mqtt (Mosquitto)
      - mosquitto_pub (producer)
      - mosquitto_sub (consumer)
    - rtl_433
      - rtl_433 -F "mqtt://localhost:1883"
      - rtl_433 -F "mqtt://localhost:1883,retain=1"
  - web
    - MQTT.fx (JavaFX)
      - Instal·lació / Installation Download
        
        rpm
        
        wget http://www.jensd.de/apps/mqttfx/1.7.1/mqttfx-1.7.1-1.x86_64.rpm
        
        Mageia
        
        urpmi mqttfx-1.7.1-1.x86_64.rpm
      - Ús / Usage
        
        Desktop MQTT Client for Adafruit.io
        
        ...
  - Python
    - mqtt-paho (Eclipse)
  - Other
    - Shelly
    - Telegraf
- Projects
  - Raspberry Pi projects with MQTT
  - Theengs OpenMQTTGateway

STOMP (Streaming Text Oriented Messaging Protocol) (wp)

to connect to message-oriented middleware
STOMP Protocol Specification, Version 1.2

Frames and headers

	frame	standard headers			request headers	response headers
		content-length	content-type	receipt
connection frames	`CONNECT`	MAY			`accept-version` `host` `login` `passcode` `heart-beat`
connection frames	`CONNECTED`	MAY				`version` `heart-beat` `session` `server`
client frames	`SEND`	MAY	SHOULD	MAY	`destination` `transaction`
	`SUBSCRIBE`	MAY		MAY	`destination` `id` `ack: [auto, client, client-individual]`
	`UNSUBSCRIBE`	MAY		MAY	`id`
	`BEGIN`	MAY		MAY	`transaction`
	`COMMIT`	MAY		MAY	`transaction`
	`ABORT`	MAY		MAY	`transaction`
	`ACK`	MAY		MAY	`id` `transaction`
	`NACK`	MAY		MAY	`id` `transaction`
	`DISCONNECT`	MAY		MAY	`receipt`
server frames	`MESSAGE`	MAY	SHOULD			`destination` `message-id` `subscription` `ack`
	`RECEIPT`	MAY				`receipt-id`
	`ERROR`	MAY	SHOULD			`receipt-id` `message`

Server
- ActiveMQ
- RabbitMQ
Client
- CLI
  - stomp (from stomp.py)
    - stomp --verbose --ssl -H b-xxxx-1.mq.eu-south-2.amazonaws.com -P 61614 --user=myuser --password=mypassword
      - subscribe /queue/my_queue
      - ...
    - stomp --verbose -H vis.musicradio.com -P 61613
      - subscribe /topic/fm/ce1/c479/09580/image
      - ...
  - telnet/ncat client (producer: SEND; consumer: SUBSCRIBE)
    - RadioDNS
    - Notes
      - to end every frame:
        
        ^@ = Ctrl+@
      - to end communication:
        
        when using ncat:
        
        (just DISCONNECT)
        
        when using telnet:
        
        ^] = Ctrl+]
    - frame (you must put an empty line between header and body and end the frame with ^@)
      - COMMAND header1:value1 header2:value2 Body^@
    - telnetncat vis.musicradio.com 61613
      - Trying 81.20.48.151... Connected to 81.20.48.151. Escape character is '^]'.
      - CONNECT ^@
      - SUBSCRIBE destination: /topic/fm/ce1/c479/09580/imageack: auto ^@
      - DISCONNECT receipt:77 ^@
- Python
  - stomp.py
    - ...

HTTP headers

List of HTTP header fields (wp)

Response header can be on html head: <meta http-equiv="content-type" content="application/vnd.hbbtv.xhtml+xml;charset=UTF-8">

	name	info	related headers	usage with curl
Request	`Accept`			`curl -H "http_request_header: value"` `curl --header"If-Modified-Since: Fri, 23 Oct 2009 19:22:47 GMT" http://...curl-H "Origin: http://..." curl -H "Host: toto.tata.org" curl -H "Cache-Control: no-cache"`
	`Accept-Language`
	`Accept-Encoding`
	`If-Modified-Since`		`Last-Modified`
	`Origin`	CORS	`Access-Control-Allow-Origin` `Access-Control-Allow-Credentials`
	`Access-Control-Request-Headers`
	`Access-Control-Request-Methods`
	`Range`
	`User-Agent`
	`Authorization`	Django authentication
	`Host`
	`Cache-Control`	Cache
	...
Response	`Access-Control-Allow-Origin`	CORS	`OriginAccess-Control-Request-Headers` `Access-Control-Request-Methods`	`curl -i http://... curl -I [-X GET] http://...`
	`Access-Control-Allow-Credentials`	CORS
	`Content-Disposition`
	`Content-Encoding`
	`Content-Length`
	`Content-Type`
	`Date`
	`Last-Modified`	HTTP-date in RFC 2616 format
	`Link`
	`Refresh`
	`Server`
	...

Request headers (Servlet tutorial)
- Showing Request Headers
An Overview of Request Headers
Response Headers (Servlet tutorial)
HTTP Client Net Debugger
Content-Disposition
- How to use content-dispositon header to determine filename
  - curl --remote-name ...
  - wget --content-disposition ...
- Adding a variable in Content disposition response file name-python/django
- Presigned URLs (AWS S3, boto)
- AWS CloudFront
  - Behaviour

HTTP request methods (wp)

method	description	note	usage		related HTML tag
			safe	idempotent
HEAD	same as GET, but without the body
GET	requests a representation of the resource	only to retrieve	y	y	href
POST	submits data to be processed		n	n	form method="post"
PUT	uploads a representation of the resource		n	y
DELETE	deletes the resource
TRACE	echoes the received request
OPTIONS	http methods supported by the server
CONNECT
PATCH	apply partial modifications to a resource

Ús / Usage

The Definitive Guide to GET vs POST
When do you use POST and when do you use GET?

CRUD

	list endpoint	detail endpoint
POST	Create	-
GET	list	Retrieve
PATCH	-	Update
DELETE	-	Delete

curl -X http_method
Django
- views
- djangorestframework routers

POST

generation		received by CGI
html form	curl	content-type	as standard input
<form action="my_cgi.sh" method="post"> <input type="text" name="tata" value="tata_value"> <input type="text" name="titi" value="titi_value"> <input type="submit"> </form>	`curl -i -X POST --data 'tata=tata_value' --data 'titi=titi_value'` `curl -i -H "Content-Type: application/x-www-form-urlencoded" -d 'tata=tata_value&titi=titi_value'` `curl -i -X POST --data-binary 'tata=tata_value' --data-binary 'titi=titi_value'`	application/x-www-form-urlencoded	tata=tata_value&titi=titi_value
<form action="my_cgi.sh" method="post" enctype="multipart/form-data"> <input type="text" name="tata" value="tata_value"> <input type="text" name="titi" value="titi_value"> <input type="submit"> </form>	`curl -i -X POST -F tata=tata_value -F titi=titi_value` `curl -i -X POST -H 'Content-Type: multipart/form-data' -F tata=tata_value -F titi=titi_value`	multipart/form-data; boundary=------------------------6e0b5ea0578a6399	--------------------------6e0b5ea0578a6399 Content-Disposition: form-data; name="tata" tata_value --------------------------6e0b5ea0578a6399 Content-Disposition: form-data; name="titi" titi_value --------------------------6e0b5ea0578a6399--
	`curl -i -X POST -H 'Content-Type: application/json' --data '{"toto":"toto_value","tata":"tata_value"}'` `url="https://127.0.0.1:8000/" my_var="toto value"; json_code="{\"toto\":\"${my_var}\"}"; curl_options="-H 'Content-Type: application/json' --data '$json_code'"; curl_command="curl -v -X POST $curl_options '$url'"; eval "$curl_command"` `my_var="toto l'educat";` `my_escaped_var=$(echo "${my_var}" \| sed "s/'/'\\\''/g"); json_code="{\"toto\":\"${my_escaped_var}\"}"; curl_options="-H 'Content-Type: application/json' --data '$json_code'"; curl_command="curl -v -X POST $curl_options '$url'"; eval "$curl_command"`	application/json

HTTP status codes

Response Status Codes (RFC 7231, Section 6)
Hyper Text Coffee Pot Control Protocol (HTCPCP/1.0) (RFC 2324)
- 418
List of HTTP status codes (wp)
Setting HTTP Status Codes (Servlet tutorial)

family	description	code
1xx	Informational	100 Continue 101 Switching Protocols 102 Processing 103 Early Hints
2xx	Success	200 OK 201 Created 202 Accepted ...
3xx	Redirection	300 Multiple Choices 301 Moved Permanently (POST is switched to GET) ... 308 Permanent Redirect (do not switch methods)
4xx	Client error	400 Bad Request 401 Unauthorized 402 Payment Required 403 Forbidden 404 Not Found 405 Method Not Allowed ... 408 Request Timeout ... 409 Conflict ... 415 Unsupported Media Type ... 418 I'm a teapot ... 429 Too Many Requests ... 451 Unavailable For Legal Reasons ... 460 The load balancer received a request from a client, but the client closed the connection with the load balancer before the idle timeout period elapsed.
5xx	Server error	500 Internal Server Error 501 Not Implemented 502 Bad Gateway 503 Service Unavailable 504 Gateway Timeout 505 HTTP Version Not Supported ...

4xx
- 499
  - Possible reason for NGINX 499 error codes
    - timeouts

DOCTYPE / DTD

Document type declaration (DOCTYPE) (wp)
Document type definition (DTD) (wp)


HTML	4.01 trans
	4.01 strict	`<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"> <html>...</html>`
	5	`<!DOCTYPE html> <html>...</html>`
XHTML
CE-HTML		`<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "ce-html-1.0-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml">...</html>`
HbbTV	1.1.1
HbbTV	1.2.1	`<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE html PUBLIC "-//HbbTV//1.2.1//EN" "http://www.hbbtv.org/dtd/HbbTV-1.2.1.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">...</html>`

HTML version information (transitional vs strict) (W3C)
Doctype switching and standards compliance: An overview
Mozilla Quirks Mode Behavior

Emmagatzematge / Storage
- Galetes / Cookies
  - Info
    - Cookie I-D Drafts
    - Cookiecentral
    - Persistent Client State HTTP Cookies (Netscape)
    - Handling Cookies (Servlet tutorial)
    - SameSite
      - SameSite cookies explained
      - SameSite cookie recipes
      - SameSite=None: Known Incompatible Clients
      - Firefox (>=69)
        
        about:config
        
        network.cookie.sameSite.laxByDefault
        
        network.cookie.sameSite.noneRequiresSecure
      - Chrome (>=80)
        
        SameSite Updates
        
        chrome://flags/#same-site-by-default-cookies
        
        chrome://flags/#cookies-without-same-site-must-be-secure
  - Implementació / Implementation
    - Navegadors
      - Firefox: about:preferences#privacy
      - Chrome: chrome://settings/content/cookies
    - curl
    - PHP
    - Javascript
      - document.cookie (MDN)
      - JavaScript Cookies (w3schools)
      - Ajax
        
        How to get a cookie from an AJAX response?
        
        server
        
        Requests with credentials
        
        set these http response headers (Access-Control-Allow-Origin:"*" is not allowed in this case):
        
        Access-Control-Allow-Origin: http...
        Access-Control-Allow-Credentials: true
        
        client (jQuery)
        
        $.ajax("http.../login", { type: "POST", ... // set cookie (from Set-Cookie response header to cookie jar): xhrFields: { withCredentials: true }, // cross domain: crossDomain: true, .... });
        $.ajax("http.../some_function", { type: "GET", ... // use cookie (from cookie jar to Cookie request header): xhrFields: { withCredentials: true }, // cross domain: crossDomain: true, .... });
        
        How to set cookie value with AJAX request?
        
        HTTP Cookies and Ajax requests over HTTPS
- Emmagatzematge DOM / DOM storage
Same origin policy (wp)
- Ways to circumvent the same-origin policy (so)
- JSONP (JSON with padding) (wp)
  - only GET
  - only from <script>
  - response is json inside a Javascript function
  - you should use CORS instead
- Cross-Origin Resource Sharing (CORS) (wp)
  - HTTP headers
  - W3C CORS
  - enable cross-origin resource sharing
  - XMLHttpRequest (Ajax)
  - HTTP access control (CORS) (dev Mozilla)
  - Credentials: cookies, HTTP authentication data
  - CORS with Javascript and PHP
  - CORS with nginx
  - CORS with curl
  - AWS
    - CORS in AWS S3
    - CORS in CloudFront
- easyXDM
- HTML5
  - postMessage
JSON-RPC
- Info
  - Specification
  - JSON-RPC over WS (Mongoose)
- Basics
  - request frame:
    - method attribute (mandatory)
    - params attribute (optional)
  - if request frame has an optional id attribute, then a server must send a response frame
  - frames without id are considered notification frames; server does not response on notification frames
  - communication is bidirectional
- Over
  - MQTT
  - HTTP
  - WebSocket

Push

Comet / Ajax push (wp)
- Categories
  - Streaming
    - hidden iframe
    - XMLHttpRequest
  - Ajax with long polling
    - XMLHttpRequest long polling
      - Simple “Long Polling” example code? (so)
        
        twisted, cometD, nodejs ...
    - Script tag long polling
- Implementations
  - APE

WebSocket (wp) (HTML5)

RFC 6455
WebSocket in Django
Websockets load-balancing with HAProxy
9 Websocket Servers for Reliable Real-time Applications
WebSocket (Full Stack Python)
- arquitectura
Escalabilitat / Scalability
- Scaling WebSockets (Ably)

Eines / Tools

nginx
- Behind a reverse proxy (socket.io)

...

format server client

raw

CLI

wscat -l 3000
websocat -s 3000
iocat -l -p 3000

Languages

ws (JavaScript)

CLI

wscat -c ws://127.0.0.1:3000
websocat ws://127.0.0.1:3000
iocat ws://127.0.0.1:3000
Weasel

Languages

ws (JavaScript)
websockets (Python)
micropython-async_websocket_client (MicroPython)

JSON-RPC

Languages

jsonrpcclient (Python)
jsonrpc-base (Python)
- jsonrpc-websocket (asyncio, aiohttp) (Python)
  - jsonrpc-requests (requests) (Python)
aioshelly (aiohttp) (Python)

socket.io

CLI

iocat --socketio -l -p 3000

Languages

socket.io (JavaScript)
python-socketio (Python)

CLI

iocat --socketio ws://127.0.0.1:3000
- as iocat is using ?EIO=3, error "Unsupported protocol version" is returned; to solve this, new (v4) socket.io server has to be started with option:
  - js:
    - const io = new Server(server, {
      allowEIO3: true // false by default
      });
  - python: ...
iocat --emit-key "chat message" --socketio ws://127.0.0.1:3000/

Languages

socket.io-client (JavaScript)
python-socketio (Python)

info instal·lació / installation docs servidor / server client

Firefox

Responses

raw
JSON
Socket.IO
...

Weasel

websocat

Rust toolchain (Mageia)
- curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
- cargo install --features=ssl websocat
...

websocat -s 1234

websocat ws://127.0.0.1:1234/
websocat ws://ws.vi-server.org/mirror
websocat --jsonrpc ...

wscat

npm install -g wscat

wscat -c ws://websocket-echo.com

plain

socket.io

overhead (not plain)
features
- HTTP long-polling fallback
- auto reconnection
- packet buffering
- acknowledgements
- broadcasting
- multiplexing

Getting started
- chat, based on express
- package.json
  - auto upadated when running npm install ...
- index.js
- index.html
Documentation (v4)

bash
- iocat
JavaScript (Node.js)
- socket.io
  - import { Server } from "socket.io"; const io = new Server(3000); io.on("connection", (socket) => { // send a message to the client socket.emit("hello", "world"); // receive a message from the client socket.on("howdy", (arg) => { console.log(arg); // prints "stranger" }); });
  - import { createServer } from "http"; import { Server } from "socket.io"; const httpServer = createServer(); const io = new Server(httpServer, { path: "/my-custom-path/" });
  - const express = require('express'); const app = express(); const http = require('http'); const server = http.createServer(app); const { Server } = require("socket.io"); const io = new Server(server); app.get('/', (req, res) => { res.sendFile(__dirname + '/index.html'); }); io.on('connection', (socket) => { console.log('a user connected'); socket.on('disconnect', () => { console.log('user disconnected'); }); socket.on('chat message', (msg) => { io.emit('chat message', msg); }); }); server.listen(3000, () => { console.log('listening on *:3000'); });
  - ...

Python

Flask-SocketIO

python-socketio

deployment	creation
aiohttp	`from aiohttp import web import socketio sio = socketio.AsyncServer() app = web.Application() sio.attach(app) async def index(request): """Serve the client-side application.""" with open('index.html') as f: return web.Response(text=f.read(), content_type='text/html')@sio.event def connect(sid, environ): print("connect ", sid) @sio.event async def chat_message(sid, data): print("message ", data) await sio.emit("chat_message", data) @sio.event def disconnect(sid): print('disconnect ', sid) app.router.add_static('/static', 'static') app.router.add_get('/', index) if __name__ == '__main__': web.run_app(app)`
WSGI	`import socketio sio = socketio.Server() app = socketio.WSGIApp(sio)`
ASGI	`# sockeio only import socketio sio = socketio.AsyncServer(async_mode="asgi") app = socketio.ASGIApp(sio) # not needed when using Daphne @sio.on("connect") async def connect(sid, env): print("New Client Connected to This id :"+" "+str(sid)) # send message to connected client await sio.emit("send_msg", "Hello from Server") # uvicorn.run(app, host='127.0.0.1', port=5000)`
	`# socketio and FastAPI # https://python-socketio.readthedocs.io/en/latest/server.html#running-as-an-asgi-application # fastapi from mywebapp import fastapi_app # a FastAPI or other ASGI application` `# socketio sio = socketio.AsyncServer(cors_allowed_origins="*", async_mode="asgi") # both socketio and fastapi global_app = socketio.ASGIApp(sio, fastapi_app)# uvicorn if __name__=="__main__": uvicorn.run("socketio_fastapi_uvicorn:global_app", host="0.0.0.0", port=7777, lifespan="on", reload=True)`
	`# socketio mounted from FastAPI # https://medium.com/@parth116.rejoice/socket-io-with-python-fast-apis-1a1956b3989a`import uvicorn import socketio from fastapi import FastAPI # fastapi app = FastAPI() # socketio sio = socketio.AsyncServer(cors_allowed_origins="*", async_mode="asgi") socket_app = socketio.ASGIApp(sio) # mount socketio from fastapi app.mount("/", socket_app) # fastapi @app.get("/") def read_root(): return {"Hello": "World"} # socketio @sio.on("connect") async def connect(sid, env): print("New Client Connected to This id :"+" "+str(sid)) @sio.on("disconnect") async def disconnect(sid): print("Client Disconnected: "+" "+str(sid)) # uvicorn if __name__=="__main__": uvicorn.run("fastapi_socketio_uvicorn:app", host="0.0.0.0", port=7777, lifespan="on", reload=True)

bash
- iocat
JavaScript (browser, Node.js, React Native)
- socket.io-client
  - <!DOCTYPE html> <html> <head> </head> <body> <script src="https://cdn.socket.io/4.8.0/socket.io.min.js"></script> <script> import {io} from "socket.io-client"; const socket = io("ws://localhost:3000") // receive a message from server socket.on("hello", (arg) => { console.log(arg); // prints "world" }); // send a message to the server socket.emit("howdy", "stranger"); </script> </body> </html>
  - import { io } from "socket.io-client"; const socket = io("https://example.com", { path: "/my-custom-path/" });
  - ... <body> <ulid="messages"></ul> <form id="form" action=""> <input id="input" autocomplete="off" /><button>Send</button> </form> <script src="https://cdn.socket.io/4.5.4/socket.io.min.js"></script> <script> var socket = io(); var messages = document.getElementById('messages'); var form = document.getElementById('form'); var input = document.getElementById('input'); form.addEventListener('submit', function(e) { e.preventDefault(); if (input.value) { socket.emit('chat message', input.value); input.value = ''; } }); socket.on('chat message', function(msg) { var item = document.createElement('li'); item.textContent = msg; messages.appendChild(item); window.scrollTo(0, document.body.scrollHeight); }); </script> </body> </html>
  - ...
Python
- python-socketio
  - import asyncio import socketio sio = socketio.AsyncClient() @sio.event async def connect(): print("Received event: connect") print("my sid is", sio.sid) @sio.event def connect_error(data): print("The connection failed!") @sio.on('*') async def catch_all(event, data): print(f"Received event {event} with data: {data}") @sio.event async def my_message(data): print("Received event: my_message, with data: ", data) await sio.emit('my response', {'response': 'my response'}) @sio.on("chat message") async def on_missatge(data): print("Received event: chat message, with data: ", data) @sio.event async def disconnect(): print("Received event: disconnect") async def main(): await sio.connect('http://localhost:3000') await sio.wait() if __name__ == '__main__': asyncio.run(main())
- ...
...

socket.io

...

server client

import

js
py

js	`<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/4.8.0/socket.io.js"></script>`
py	`import socketio`

init

standard
- sio = socketio.Server()
asyncio
- sio = socketio.AsyncServer(async_mode="asgi")

js const sio = io("ws://localhost:7777/");

standard
- sio = socketio.Client() sio.connect("http://localhost:7777")

asyncio
- sio = socketio.AsyncClient() async def main(): await sio.connect("http://localhost:7777") await sio.wait()

receive connection

js
py	standard asyncio `@sio.event async def connect(sid, environ, auth): pass`

js
py	standard asyncio `@sio.event async def connect(): pass`

send message

standard
- to all clients:
  - sio.emit(...)
- to a single user
  - sio.emit('my_event_name', {'data': 'foobar'}, to=user_sid)
- to a single client, with callback:
  - def my_callback(): print("callback invoked!") sio.emit('my event', {'data': 'foobar'}, to=user_sid, callback=my_callback)
- with acknowledgement
  - sio.call("my_event_name", {'data': 'foobar'}, to=user_sid)
- using rooms
  - ...
  - skipping a single session_id (e.g. emitter)
    - sio.emit('my reply', data, room='chat_users', skip_sid=sid)
asyncio
- await sio.emit("my_event_name", "Hello world")

js
py	standard `sio.emit("my_event_name", "tata")` asyncio `await sio.emit("my_event_name", {"response": "my response"})`

receive message

js
py	standard asyncio `@sio.event async def my_event_name(sid, data): pass` `@sio.on("my_event_name") async def my_function(sid, msg, data): pass` `@sio.on('*') def any_event(event, sid, data): pass`

js	`sio.on('send_msg',(data)=>{ console.log(data); } );`
py	standard asyncio `@sio.event async def my_event_name(data): pass` `@sio.on("my_event_name") async def on_my_missatge(data): pass`

initiate disconnection

js
py

js
py	sio.disconnect() await sio.disconnect()

receive disconnection

js
py	`@sio.event async def disconnect(sid, reason): pass`

js
py	`@sio.event async def disconnect(): pass`

Media types (ftp)
Byte articles
- Website API Discovery

Cache

HTTP header	cache on servers
	Apache	Nginx
`Cache-Control`
`Expires`

Understanding HTTP Caching
- conditional request: If-Modified-Since
Caching Tutorial for Web Authors and Webmasters
Using nocache meta tags with JSPs
What's the difference between Cache-Control: max-age=0 and no-cache?

vCard and vCalendar
MIME-Types
- IANA MIME Media Types
- HTTP servers configuration
  - MIME type (SVG wiki)
  - Internet media type (wp)
  - Apache
  - IIS
    - HTTP headers / File types
Imatges / Images
- Image optimization | Web fundamentals (Google Developers)
  - PNG, GIF, JPEG, WebP
- A guide for creating a better retina web
- retina.js

Proxy

Info
- Proxy vs reverse proxy
  - [direct] proxy: una petició iniciada des d'un client en una subxarxa privada cap a Internet pot passar per un proxy, que va a buscar la resposta a Internet i li retorna cap al client
    - exemples: Squid, ...
  - reverse proxy: una petició iniciada des d'un client a Internet cap a un servidor, passa per un proxy invers, que va a buscar la resposta en un altre servidor (normalment en una subxarxa interna) i li retorna la resposta cap al client a Internet
    - tasques: balanceig de càrrega, autenticació, desxifratge, cache
    - exemples: nginx, ...
General
- Filtering (Programsorg)
- Sorm: Proxy source code (ijs)
- Cameleon
- Proxylist
- Rosinstrument
- Djsyndrome
- Proxys4all
Articles
- SSL Proxying (Byte article)
Programari / Software
- Proxomitron
- Web Filter
- Muffin
- Alfajor
- Junkbuster (C)
- Alstone (Java) (ijs)
- HTTPort
- Remote Controlled Caching Proxy Server
Server
- Squid proxy server
  - How To: Configure Squid Proxy Server
  - Squid setup
Client
- configuració / setup
  - bash environment variable
    - export http_proxy=http://server-ip:port/
    - export http_proxy=http://USERNAME:PASSWORD@server-ip:port/
  - GUI
    - Centre de control de Mageia
      - Xarxa i internet
        
        Servidor intermediari
- ús / usage
  - CLI
    - wget http://toto.org/page.html
    - wget --proxy-user=USERNAME --proxy-password=PASSWORD http://toto.org/page.html
    - curl http://toto.org/page.html
    - curl --proxy-user user:password http://toto.org/page.html
  - aplicacions / applications
    - Firefox
      - Preferències -> Paràmetres de xarxa
    - Chrome
      - Configuració -> Sistema -> Obre la configuració del servidor intermediari de l'ordinador
Discover
- FindProxyForURL
- xfindproxy

Servidors HTTP / HTTP servers

Comparison of web servers (wp)
Comparison of lightweight web servers (wp)
Best Practices for Speeding Up Your Web Site (Steve Sounders)
Servidors HTTPS / HTTPS server
- Apache secure server (HTTPS)
Balanceig de càrrega / Load balancing
- Django deployment: scaling
- Solucions / Solutions
  - DNS
  - Maquinari / Hardware
  - Programari / Software
    - Apache mod_proxy_balancer
    - Perlbal
    - HAProxy (wp) (OSI level 7)
    - Linux Virtual Server (OSI level 4)
Cache on servers
- Cache on Apache
- Cache on Nginx
- Cache on Django
- Memcached
  - Memcached with Django
  - Debug
    - /etc/sysconfig/memcached
      - ... OPTIONS="-vv"
    - sudo systemctl restart memcached.service
    - tail -n 200 -f /var/log/messages | grep memcached
      - get ...
        END
        set ...
        STORED
        ...
        get ...
        END
        ...
        (after cache period)
        get ...
        END
        set ...
        STORED
  - Mageia (templated service)
    - systemctl start memcached@11211.service
Servidors de prova / Test servers
- Django http server
- Python
  - How to Launch an HTTP Server in One Line of Python Code
  - cd /path/with/docs/ python -m http.server 8000
  - Python 2: python -m SimpleHTTPServer 8000
- PHP
  - php -S 0.0.0.0:8000
- http-server (NodeJS)
  - Installation
    - install nvm
    - [nvm use ...]
    - npm install -g http-server
  - Usage
    - http-server -p 8080 /usr/share/janus/demos
    - proxy non-resolved requests to another server
      - http-server -p 8080 /usr/share/janus/demos -P http://192.168.2.69:8000
    - https
      - creeu un certificat autosignat / create a self-signed certificate:
        
        openssl req -new -nodes -keyout <your_ip_address>.key -sha256 -x509 -days 365 -out<your_ip_address>.crt
        
        Common name: <your_ip_address>
      - http-server --ssl --cert <your_ip_address>.crt--key<your_ip_address>.key

Nginx (nginx.com) (wp)

Compilació / Compilation
- Init scripts
- show compilation options
  - nginx -V
- check config
  - nginx -t
Instal·lació / Installation
- Mageia
  - urpmi nginx
  - sytemctl enable nginx
  - sytemctl start nginx
- Ubuntu
- CentOS
  - How To Install Nginx on CentOS 7
  - LEMP server on CentOS 7 with FastCGI
  - sudo yum install epel-release
  - sudo yum install nginx
  - Official nginx repositories for CentOS
    - /etc/yum.repos.d/nginx.repo
  - sudo systemctl start nginx.service
  - Firewall (only needed if firewalld.service is running)
    - sudo firewall-cmd --permanent --zone=public --add-service=http sudo firewall-cmd --permanent --zone=public --add-service=https sudo firewall-cmd --reload
  - SELinux
    - SELinux on HTTP servers
    - sudo setsebool -P httpd_read_user_content 1
    - if only using nginx (not uwsgi)
      - Linux (Centos) and Nginx permissions on another disk
        
        sudo chcon -v -R --type=httpd_sys_content_t <www-root>
      - Upgrade to 6.6 Changes Nginx/SELinux Interaction?
    - if using also uwsgi (unix socket permission denied)
      - SELinux policy for nginx and GitLab unix socket in Fedora 19
      - disable Selinux:
        
        setenforce 0
      - detect the problem
        
        access your website, to generate logs
        
        sudo yum install policycoreutils-python
        
        sudo grep nginx /var/log/audit/audit.log | audit2allow
      - solve the problem (do not use -M option, as you will not be able to edit the te text file to add rules)
        
        sudo -i
        
        grep nginx /var/log/audit/audit.log | audit2allow-m nginx > nginx.te
        
        checkmodule -M -m -o nginx.mod nginx.te
        
        semodule_package -o nginx.pp -m nginx.mod
        
        semodule -i nginx.pp
        
        setenforce 1
      - example
        
        nginx.te
        
        module nginx 1.0; require { type httpd_t; type init_t; type nfs_t; type user_home_t; type var_lib_t; type unlabeled_t; class unix_stream_socket connectto; class file {read getattr open}; class lnk_file read; class sock_file write; } #============= httpd_t ============== allow httpd_t init_t:unix_stream_socket connectto; allow httpd_t nfs_t:file {read getattr open}; allow httpd_t nfs_t:lnk_file read; allow httpd_t user_home_t:file {read open}; allow httpd_t var_lib_t:sock_file write; allow httpd_t unlabeled_t:lnk_file read; allow httpd_t unlabeled_t:file {read open};
    - Problems / Problemes
- Django wsgi
  - Django: nginx + uWSGI
  - Django: nginx + gunicorn

Documentation (.org)

Beginner's Guide

Modules

nginx.org nginx.com

3rd party

core

ngx_http_core_module

location
root
Embedded variables
- $request
- $request_length
- $request_time
- $request_body_file
  - passed to proxy, fastcgi, uwsgi, scgi

ngx_http_headers_module

add_header (any header)
expires (cache headers)

	examples	HTTP headers
add_header
expires	`# at now + time (>=0) expires 1d;`	`Date: Thu, 23 Apr 2020 15:34:53 GMT Last-Modified: Thu, 23 Apr 2020 15:19:47 GMT Expires: Fri, 24 Apr 2020 15:34:53 GMT Cache-Control: max-age=86400`
	`# at now + time (<0) expires -1d;`	`Date: Thu, 23 Apr 2020 15:36:14 GMT Last-Modified: Thu, 23 Apr 2020 15:19:47 GMT Expires: Wed, 22 Apr 2020 15:36:14 GMT Cache-Control: no-cache`
	`# at file modification + time expires modified 1d;`	`Date: Thu, 23 Apr 2020 15:37:35 GMT Last-Modified: Thu, 23 Apr 2020 15:19:47 GMT Expires: Fri, 24 Apr 2020 15:19:47 GMT Cache-Control: max-age=seconds_remaining_to_23_Apr_2020:15:19:47_GMT`
	`# at precise date (local time) expires @15h30m;`	`Date: Thu, 23 Apr 2020 15:39:34 GMT Last-Modified: Thu, 23 Apr 2020 15:19:47 GMT Expires: Fri, 24 Apr 2020 13:30:00 GMT Cache-Control: max-age=seconds_remaining_to_24_Apr_2020:13:30:00_GMT`
	`# at 1st Jan 1970 expires epoch;`	`Expires: Thu, 01 Jan 1970 00:00:01 GMT` `Cache-Control: no-cache`
	`# at 31st Dec 2037 (max-age: 10 years) expires max;`	`Expires: Thu, 31 Dec 2037 23:55:55 GMT` `Cache-Control: max-age=315360000`
	`# no cache headers expires off;`	`Expires Cache-Control`

ngx_http_rewrite_module

return

proxy

ngx_http_proxy_module

proxy_pass

fastCGI

ngx_http_fastcgi_module

fastcgi_pass

authentication

Access

ngx_http_access_module

deny
allow

Basic

ngx_http_auth_basic_module

Basic HTTP Authentication with Nginx

JWT ngx_http_auth_jwt_module

Digest ngx_http_auth_digest

PAM ngx_http_auth_pam_module

Request ngx_http_auth_request_module ngx_http_auth_request_module

languages LUA ngx_http_lua_module

image resize ngx_http_image_filter_module

...

Service setup
- NGINX systemd service file
- nginx.service
  - # https://www.nginx.com/resources/wiki/start/topics/examples/systemd/ [Unit] Description=The NGINX HTTP and reverse proxy server After=syslog.target network.target remote-fs.target nss-lookup.target cloud-init.service [Service] Type=forking PIDFile=/run/nginx.pid TimeoutStartSec=200s ExecStartPre=/usr/sbin/nginx -t PrivateTmp=true # other ExecStartPre scripts # ExecStartPre=/bin/bash -c 'for script in /etc/nginx/execstartpre.d/*; do $script; done' ExecStartPre=/bin/bash -c 'if [[ -d /etc/nginx/execstartpre.d && -n "$(/usr/bin/ls -A /etc/nginx/execstartpre.d)" ]]; then for script in /etc/nginx/execstartpre.d/*; do $script; done; else echo "/etc/nginx/execstartpre.d/ is an empty dir"; fi' ExecStart=/usr/sbin/nginx ExecReload=/bin/kill -s HUP $MAINPID ExecStop=/bin/kill -s QUIT $MAINPID [Install] WantedBy=multi-user.target
- /tmp
  - /usr/lib/systemd/system/nginx.service
    - [Service] ... PrivateTmp=true
  - real /tmp dir for nginx is:
    - /tmp/systemd-private-...-nginx.service-.../tmp

Configuració / Configuration

How To Optimize Nginx Configuration
- Timeouts
Modificació del fitxer de configuració / Modification of config file
- Python
  - Any good way to programmatically change nginx config file from python?
  - letsencrypt-nginx
  - pynginxconfig (wiki) (Python) (gc)
    - pynginxconfig (gh)
    - pynginxconfig-new (gh)
  - nginxparser (Python, uses Pyparsing)
    - pip install git+https://github.com/fatiherikli/nginxparser.git
- Javascript
  - nginx-conf (npm)
Variables
- Alphabetical index of variables
- ...

Modular


`worker_processes number \| auto;`
`events { ... }`	`worker_connections number;`
`# ngx_http_core_module http { ... }`	`include file \| mask; include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; gzip on;`
	`# ngx_http_upstream_module upstream name { ... }`	`# round-robin server address [parameters]; server 127.0.0.1:8001; server 127.0.0.1:8002;`
	`# other server{} sections in files: include /etc/nginx/conf.d/*.conf;`
	`server { ... }`	`listen address[:port] ...; listen port ...; listen unix:path ...; server_name name ...; #see: regex listen 80; server_name localhost; access_log` `/var/log/nginx/proxy.access.log; error_log /var/log/nginx/proxy.error.log;`
		`# ngx_http_ssl_module ssl_certificate file; ... ssl_certificate /etc/letsencrypt/live/www.example.org/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/www.example.org/privkey.pem; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers HIGH:!aNULL:!MD5;`
		`# nginx-vod-module: vod settings vod_mode ...; vod_upstream_location ...; vod_fallback_upstream_location ...; vod_last_modified ...; vod_last_modified_types ...; # nginx-vod-module: vod caches ... # nginx-vod-module: file handle caching / aio ...`
		`# other location{} include ...*.conf;`
		`location [ = \| ~ \| ~* \| ^~ ] uri { ... }` = exact matching ~ case-sensitive matching regex ~* case-insensitive matching regex ^~ ... `location @name { ... } location / { ... }`	`root ...; root html;`
			`alias ...;`
			`# ngx_http_proxy_module proxy_pass ...; proxy_read_timeout ...; proxy_send_timeout ...; proxy_http_version ...; proxy_set_header ...; proxy_redirect ...;`
			`# ngx_http_fastcgi_module fastcgi_pass ...; fastcgi_param ...;`
			`# ngx_http_uwsgi_module uwsgi_pass ...;`
		`location = /50x.html {}`	`root html;`
`# nginx-rtmp-module rtmp{}`	`include /etc/nginx/rtmp.conf.d/*.conf;`
	`server{}`	`listen 1935;`
		`application ... {}`	`live on; wait_video on; wait_key on;`

Regex (PCRE - Perl regular expressions)

examples

server

Wildcard names

server_name .example.org;
- matches: example.org, *.example.org

asterisk only at the begin or end:

server_name *.example.org;
server_name my.example.*;

asterisk not at the begin nor end is not allowed:

www.*.example.org (use regex instead: ~^www\..+\.example\.org$)
w*.example.org (use regex instead: ~^w.*\.example\.org$)

Regular expressions names
(start with ~)

server_name ~^www\d+\.example\.net$;
a regular expression containing the characters “{” and “}” should be quoted:
- server_name "~^(?<name>\w\d{1,3}+)\.example\.net$";
digital captures
- usage:
  - server { server_name ~^(www\.)?(.+)$; location / { root /sites/$2; } }
named regular expression
- syntax:
  - ?<name> Perl 5.10 compatible syntax, supported since PCRE-7.0
  - ?'name' Perl 5.10 compatible syntax, supported since PCRE-7.0
  - ?P<name> Python compatible syntax, supported since PCRE-4.0
- usage:
  - server { server_name ~^(www\.)?(?<domain>.+)$; location / { root /sites/$domain; } }
...

location

Regular expressions

(empty) starting with
= exact matching
~ case-sensitive matching regex
~* case-insensitive matching regex
^~ ...

location = / { [ configuration A ] }
location / { [ configuration B ] }
location /documents/ { [ configuration C ] }
location ^~ /images/ { [ configuration D ] }
location ~* \.(gif|jpg|jpeg)$ { [ configuration E ] }

/etc/nginx/
- nginx.conf
  - user nginx; worker_processes 1; http { include /etc/nginx/mime.types; default_type application/octet-stream; # Load modular configuration files from the /etc/nginx/conf.d directory. # See http://nginx.org/en/docs/ngx_core_module.html#include # for more information. include /etc/nginx/conf.d/*.conf; }# nginx-rtmp-module
    rtmp { # Load modular configuration files from the /etc/nginx/rtmp.conf.d directory. include /etc/nginx/rtmp.conf.d/*.conf; }
- execstartpre.d/
  - script1_execstartpre.sh
  - ...
- conf.d/
  - http_server_1.conf
    - server { listen 80; server_name localhost; # Load modular configuration files from the /etc/nginx/conf.d/server_1.d/ directory. include /etc/nginx/conf.d/http_server_1.d/*.http_server.conf; }
  - http_server_1.d/
    - dir_1_a.http_server.conf
      - location /dir_1_a { alias /path/to/dir_1_a; }
    - rtmp_server_1.http_server.conf (if using nginx-rtmp-module statistics and control)
      - # rtmp statisticslocation /stat { rtmp_stat all; rtmp_stat_stylesheet stat.xsl; } location /stat.xsl { root /path/to/parent/dir/of/stat/xsl/file; } # rtmp control location /control { rtmp_control all; }
  - https_server_2.conf
    - server { # https listen 443 ssl; server_name www.example.org; ssl_certificate /etc/letsencrypt/live/www.example.org/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/www.example.org/privkey.pem; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers HIGH:!aNULL:!MD5; ... }
  - https_server_2.d/
    - ...
- rtmp.conf.d
  - rtmp_server_1.conf
    - server { listen 1935; ping 30s; notify_method get; # compatibility with GStreamer: publish_time_fix off; # Load modular configuration files from the /etc/nginx/rtmp.conf.d/wct-rtmp/ directory. include /etc/nginx/rtmp.conf.d/rtmp_server_1.d/*.rtmp_server.conf; }
  - rtmp_server_1.d/
    - app_1_a.rtmp_server.conf
      - application app_1_a { live on; wait_video on; wait_key on; }
  - rtmp_server_2.conf
    - ...
  - rtmp_server_2.d/
    - ...

HTTPS
- Configuring HTTPS servers
- Client Side Certificate Auth in Nginx
- SNI (several domains with several certificates)
  - Server Name Indication
  - check compatibility:
    - # nginx -V ... TLS SNI support enabled
  - nginx.conf
    - ...
- HSTS
  - How to Make a Website Secure Again. Adding HSTS Response Header in NGINX
  - add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
- Protocols
- Ciphers
  - ssl_ciphers
  - openssl ciphers
- nginx.conf
  - server { # https listen 443 ssl; server_name www.example.org; ssl_certificate /etc/letsencrypt/live/www.example.org/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/www.example.org/privkey.pem; #ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_protocols TLSv1.2; ssl_ciphers HIGH:!aNULL:!MD5; # hsts add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always; ...

Autenticació / Authentication
- Authentication? Why nginx?
Debug
- A debugging log (nginx.org)
  - Debugging log for selected clients
  - Logging to a cyclic memory buffer
- Debugging NGINX (nginx.com)
- when building
  - ./configure --with-debug ...
  - check it:
    - nginx -V 2>&1 | grep -- '--with-debug'
- nginx.conf
  - error_log /path/to/log debug;
  - error_log /var/log/nginx/error.log debug;
- level possible values: debug, info, notice, warn, error, crit, alert, emerg
- tail -n 200 -f /var/log/nginx/error.log
- echo variables
  - How to output variable in nginx log for debugging
  - echo-nginx-module
    - ...

Logs

Logging and monitoring
Error log
- error_log
Access log
- log_format
  - default nginx.conf
    - http { ... log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; ... }
  - add request_time:
    - http { ... log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for" $request_time'; ... }
- access_log

access.log columns when using awk

BEGIN
                          {FPAT="([^ ]+)|(\"[^\"]+\")"};

$0	all line
$1	IP address
$2	-
$3	-
$4	datetime
$5	timezone
$6	request
$7	response code
$8	bytes
$9	referer
$10	user_agent
$11	x_forwarded_for (real origin ip address)
$12	(request_time)

count (uniq -c) the most requested urls ($6):
- awk 'BEGIN {FPAT="([^ ]+)|(\"[^\"]+\")"}; {print $6}' /var/log/nginx/access.log | sort | uniq -c | sort -rn
count (uniq -c) the most requested urls ($6) inside a specified time ($4) period:
- awk 'BEGIN {FPAT="([^ ]+)|(\"[^\"]+\")"}; $4 ~ /14\/Dec\/2021:13:5/ {print $6}' /var/log/nginx/access.log | sort | uniq -c | sort -n
get requests with code ($7) 4xx and print time ($4), code ($7), request ($6):
- awk 'BEGIN {FPAT="([^ ]+)|(\"[^\"]+\")"}; $4 ~ /17\/Dec\/2021/ && $7 ~ /^4/ {print $4,$7,$6}' /var/log/nginx/access.log
get requests to given path (index.m3u8), print response code ($7) and request ($6), inside a specified time ($4) period, and sort it by number of response code
- awk 'BEGIN {FPAT="([^ ]+)|(\"[^\"]+\")"}; $4 ~ /10\/Feb\/2025:15/ && $6 ~ /index.m3u8/ {print $7,$6}' /var/log/nginx/access.log | sort | uniq -c | sort -n
get requests (from a gzipped file) sorted by transferred bytes ($8):
- zcat access.log-20241229.gz | awk 'BEGIN {FPAT="([^ ]+)|(\"[^\"]+\")"}; {print $8,$0}' | sort -n
get requests, sorted by response time (verify that you added it in log):
- grep -E "10/Mar/2025" /var/log/nginx/access.log | awk '{print $4,$7,$15}' | sort -V -k3

Monitoratge / Monitoring
- ngxtop
  - Instal·lació / Installation
    - pip install ngxtop
  - Ús / Usage
    - only last entries (live)
      - ngxtop
    - all log content
      - ngxtop --no-follow -i 'status >= 500' print request status http_referer
Headers
- your.conf
  - location /toto { add_header x-peticio $request; add_header x-temps $request_time; add_header x-longitud $request_length; return 200 'tot bé\n'; }

Reverse proxying to:

Django deployment
Nginx: the High-Performance Web Server and Reverse Proxy

nginx config	backend server			gateway interface	apps
	memcached
`# round robin upstream python_servers { server 127.0.0.1:8001; server 127.0.0.1:8002; } server { listen 127.0.0.1:8000; server_name proxy; access_log /var/log/nginx/proxy.access.log; error_log /var/log/nginx/proxy.error.log; location / { proxy_pass http://python_servers; } }`	ngx_http_proxy_module		server.py `import sys,BaseHTTPServer as B class Handler(B.BaseHTTPRequestHandler): def do_GET(self): self.wfile.write("Served from port %s" % port) def log_message(self, *args): pass if __name__ == '__main__': host,port = sys.argv[1:3] server = B.HTTPServer((host,int(port)), Handler) server.serve_forever()` `# python /tmp/server.py 127.0.0.1 8001 & # python /tmp/server.py 127.0.0.1 8002 &`
`location /mywebsocket { proxy_read_timeout 1d; proxy_send_timeout 1d; proxy_pass http://localhost:8123; #proxy_pass http://unix:/tmp/unixsocket_websocat; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; }`		websocket	`websocat -s 8123` socket.io `node index.js` `python server.py` `sio = socketio.Server()` `sio = socketio.AsyncServer()` ...
	SCGI
`upload_pass ...`	upload-module
`location ~ \.php$ { include /etc/nginx/fcgi_params; #or whatever you named it fastcgi_pass 127.0.0.1:9000; }`	FastCGI	PHP	`php-cgi -b 127.0.0.1:9000`	CGI	Bash cgi
`server { ... location /myscript { fastcgi_pass unix:/var/run/fcgiwrap.socket; # Fastcgi parameters, include the standard ones include /etc/nginx/fastcgi_params; fastcgi_param SCRIPT_FILENAME /usr/local/bin/myscript.sh; } }`	FastCGI	FCGIWrap	`systemctl start spawn-fcgi`
`# the upstream component nginx needs to connect to upstream django_mysite { server unix:///var/lib/uwsgi/mysite.sock; } # configuration of the server server { ... # Finally, send all non-media requests to the Django server. location / { uwsgi_pass django_mysite; include /etc/uwsgi/uwsgi_params } }`	UWSGI		`bin/uwsgi`	WSGI	Django wsgi.py Socket.IO WSGI `python wsgi_server.py` `import socketio sio = socketio.Server() app = socketio.WSGIApp(sio)`
	Gunicorn			WSGI
	Unit		config `{ "listeners":{ "*:8080":{ "pass":"applications/webapp" } }, "applications":{ "webapp":{ "type":"python 3", "path":"/www/", "module": "wsgi", "callable": "app" } } }`	WSGI	Flask webapp/wsgi.py `from flask import Flask app = Flask(__name__) @app.route('/') def hello_world(): return 'Hello, World!'`
upstream channels-backend { # use 127.0.0.1 instead of localhost to avoid errors related to ipv6 server 127.0.0.1:9000; } server { ... location / { try_files $uri @proxy_to_app; } location @proxy_to_app { proxy_pass http://channels-backend; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $server_name; } }	Daphne		`daphne -p 9000 myproject.asgi:application` `daphne -b 0.0.0.0 -p 9000 myproject.asgi:application` daphne -e ssl:443:privateKey=key.pem:certKey=crt.pem my_project.asgi:application systemctl start my_django_project.daphne.service `[Unit] Description=Daphne daemon (my_project) After=network.target` `[Service] PIDFile=/run/daphne/pid User=root Group=root WorkingDirectory=/path/to/my_project ExecStart=/path/to/my_project/env/bin/daphne --bind 0.0.0.0 --port 9000 --verbosity 0 my_project.asgi:application ExecReload=/bin/kill -s HUP $MAINPID ExecStop=/bin/kill -s TERM $MAINPID Restart=on-abort PrivateTmp=true LimitNOFILE=32768` `[Install] WantedBy=multi-user.target` `daphne -p 3000 asgi_server:app` daphne -b 0.0.0.0 -p 3000`asgi_server:app` systemctl start asgi_server.daphne.service ...	ASGI	FastAPI my_fastapi.py `from fastapi import FastAPI import uvicorn app = FastAPI() @app.post("...") async def post(...): ...` Django (>2 with channels or >3) my_project/my_project/asgi.py `import os import django from channels.routing import get_default_application os.environ.setdefault("DJANGO_SETTINGS_MODULE", "my_project.settings") django.setup() application = get_default_application()` Socket.IO ASGI `python asgi_server.py` `import socketio sio = socketio.AsyncServer() app = socketio.ASGIApp(sio)`
	Uvicorn		`uvicorn my_fastapi:app --reload --port 5000`	ASGI
			(uvicorn is called from python file)	ASGI	Socket.IO served by uvicorn socketio_uvicorn.py import uvicorn import socketio # socketio sio = socketio.AsyncServer(cors_allowed_origins="", async_mode="asgi") socket_app = socketio.ASGIApp(sio) # socketio @sio.on("connect") async def connect(sid, env): print("New Client Connected to This id :"+" "+str(sid)) @sio.on("disconnect") async def disconnect(sid): print("Client Disconnected: "+" "+str(sid)) # uvicorn if __name__=="__main__": uvicorn.run("socketio_uvicorn:socket_app", host="0.0.0.0", port=7777, lifespan="on", reload=True) FastAPI served by uvicorn fastapi_uvicorn.py `from fastapi import FastAPI import uvicorn app = FastAPI() @app.post("...") async def post(...): ... # uvicorn if __name__ == "__main__": uvicorn.run(app, host="0.0.0.0", port=8080)` FastAPI (with mounted Socket.IO) served by uvicorn fastapi_socketio_uvicorn.py import uvicorn import socketio from fastapi import FastAPI # fastapi app = FastAPI() # socketio sio = socketio.AsyncServer(cors_allowed_origins="", async_mode="asgi") socket_app = socketio.ASGIApp(sio) # mount socketio from fastapi app.mount("/", socket_app) # fastapi @app.get("/") def read_root(): return {"Hello": "World"} # socketio @sio.on("connect") async def connect(sid, env): print("New Client Connected to This id :"+" "+str(sid)) @sio.on("disconnect") async def disconnect(sid): print("Client Disconnected: "+" "+str(sid)) # uvicorn if __name__=="__main__": uvicorn.run("fastapi_socketio_uvicorn:app", host="0.0.0.0", port=7777, lifespan="on", reload=True)
`http { upstream unit_backend { server 127.0.0.1:8080; } server { location /unit/ { proxy_pass http://unit_backend/; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } } }` http { server { location /unit/ { # Note the trailing slash on the proxy_pass. # It tells nginx to replace /unit/ with / when passing the request, including queryparams # if required queryparms were not received by unit, an error 422 would rise proxy_pass http://127.0.0.1:8080/; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } } }	Unit		config `{ "listeners":{ "*:8080":{ "pass":"applications/fastapi" } }, "applications":{ "fastapi":{ "type":"python 3.12", "path":"/www/", "module": "asgi", "callable": "app" } } }`	ASGI	FastAPI fastapi/asgi.py `from fastapi import FastAPI` `app = FastAPI()` `@app.get("/") async def root(): return {"message": "Hello, World!"}`

AWS CloudFront
- Problems
  - 504 timeout
    - cloudfront updated the dns entries but nginx reverse proxy is still pointing to old values
    - Using DNS for Service Discovery with NGINX and NGINX Plus (official f5.com blog)
    - Nginx Reverse Proxy DNS Issue
    - Re: Does Nginx honor DNS TTLs for proxy upstreams?
      - "domain names statically configured in config are only looked up once on startup (or configuration reload)."
    - does nginx (1.20 or newer) re-resolve DNS for proxy_pass?
    - Solució / Solution
      - nginx reload
      - use variables in proxy_pass directive and resolver to force dns lookup (cannot be done when using an upstream server group upstream):
        
        resolver 10.0.0.2; # add valid=10s to ignore ttl server { location / { set $backend_servers backends.example.com; proxy_pass http://$backend_servers:8080; } }
  - nginx debug: SSL_do_handshake() failed (SSL: error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:SSL alert number 40) while SSL handshaking to upstream,
    - Solution:
      - upstream my_upstream { server my-cdn.cloudfront.net:443; } server { proxy_pass https://my_upstream/$1; proxy_ssl_server_name on; proxy_ssl_name "my-cdn.cloudfront.net"; }
    - info that led to the solution
FastCGI (CGI)
- fastcgi module (ngx_http_fastcgi_module)
- Understanding and Implementing FastCGI Proxying in Nginx
- FastCGI example
- fastcgi_params Versus fastcgi.conf – Nginx Config History
- nginx - How to run a shell script on every request?
- FCGIWrap (github)
  - Documentation
    - Child process not detached #40
  - Installing FcgiWrap and Enabling Perl, Ruby and Bash Dynamic Languages on Gentoo LEMP
  - Installation
    - Ubuntu
      - sudo apt-get install fcgiwrap
      - (no need to install spawn-fcgi, as fcgiwrap already comes with a service)
      - sudo systemctl status fcgiwrap.service
    - Mageia
      - from repo
        
        ...
      - from source
        
        install dependencies
        
        sudo dnf install libfcgi-devel
        
        download and compile
        
        git clone http://github.com/gnosek/fcgiwrap.git
        
        cd fcgiwrap
        
        autoreconf -i
        
        ./configure
        
        make
        
        sudo make install
        
        /usr/local/sbin/fcgiwrap
      - install and setup spawn-fcgi (needed?)
        
        install
        
        sudo dnf install ...
        
        setup
        
        ...
    - CentOS
      - from repo
        
        sudo dnf install fcgiwrap
      - from source
        
        14 Install Nginx, PHP5 (PHP-FPM), And Fcgiwrap
        
        Serving CGI Scripts With Nginx On CentOS 6.0 - Page 2
        
        Passos / Steps
        
        install dependencies
        
        sudo yum -y install gcc autoconf automake
        
        sudo yum -y install git
        
        sudo yum -y install fcgi-devel
        
        if you did not installed nginx from source:
        
        sudo yum -y install nginx-all-modules
        
        download and compile fcgiwrap
        
        cd ~src
        
        git clone git://github.com/gnosek/fcgiwrap.git
        
        cd fcgiwrap
        
        autoreconf -i
        
        ./configure
        
        make
        
        sudo make install
        
        /usr/local/sbin/fcgiwrap
      - install and setup spawn fcgi
        
        install
        
        sudo yum -y install spawn-fcgi
        
        setup
        
        /etc/sysconfig/spawn-fcgi
        
        # You must set some working options before the "spawn-fcgi" service will work. # If SOCKET points to a file, then this file is cleaned up by the init script. # # See spawn-fcgi(1) for all possible options. # # Example : #SOCKET=/var/run/php-fcgi.sock #OPTIONS="-u apache -g apache -s $SOCKET -S -M 0600 -C 32 -F 1 -P /var/run/spawn-fcgi.pid -- /usr/bin/php-cgi" FCGI_SOCKET=/var/run/fcgiwrap.socket # if fcgiwrap was compiled from source #FCGI_PROGRAM=/usr/local/sbin/fcgiwrap # if fcgiwrap was installed from repoFCGI_PROGRAM=/usr/sbin/fcgiwrap FCGI_USER=nginx FCGI_GROUP=nginx FCGI_EXTRA_OPTIONS="-M 0770" # IMPORTANT: -F 1 means only one worker! OPTIONS="-u $FCGI_USER -g $FCGI_GROUP -s $FCGI_SOCKET -S $FCGI_EXTRA_OPTIONS -F 1 -P /var/run/spawn-fcgi.pid -- $FCGI_PROGRAM"
        
        to redirect stderr logs to /var/log/nginx/error.log (they will be shown as "FastCGI sent in stderr:")
        
        ...OPTIONS="-u $FCGI_USER -g $FCGI_GROUP -s $FCGI_SOCKET -S $FCGI_EXTRA_OPTIONS -F 1 -P /var/run/spawn-fcgi.pid -- $FCGI_PROGRAM -f"
        
        start
        
        systemctl enable spawn-fcgi
        
        systemctl start spawn-fcgi
        
        systemctl status spawn-fcgi
        
        ...
  - Problemes / Problems
    - nginx problems related to fcgiwrap
    - nginx + fastcgi error when uploading a file over ~60kb
    - User nginx does not have /usr/local/bin in PATH variable
      - PATH issues for init.d scripts on startup
      - Solució / Solution
        
        set PATH in the first called script
  - Security issues
    - Setting up PHP-FastCGI and nginx? Don’t trust the tutorials: check your configuration!
  - Example to execute a script specified in the url
    - setup
      - /etc/nginx/default.d/toto.conf
        
        location /cgi-bin/ { # Disable gzip (it makes scripts feel slower since they have to complete # before getting gzipped) gzip off; # Set the root to /usr/lib (inside this location this means that we are # giving access to the files under /usr/lib/cgi-bin) root /usr/share/nginx; # Fastcgi socket fastcgi_pass unix:/var/run/fcgiwrap.socket; # Fastcgi parameters: include the standard ones include /etc/nginx/fastcgi_params; #Fastcgi parameters:adjust non standard parameters (e.g. SCRIPT_FILENAME) fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; #Fastcgi parameters:user-defined parameters fastcgi_param TOTO_PARAM $request_length;
        }
      - sudo systemctl reload nginx.service
    - scripts
      - mkdir -p /usr/share/nginx/cgi-bin
      - perl script
        
        /usr/share/nginx/cgi-bin/hello_world.cgi
        
        #!/usr/bin/perl -w # Tell perl to send a html header. # So your browser gets the output # rather then <stdout>(command line # on the server.) print "Content-type: text/html\n\n"; # print your basic html tags. # and the content of them. print "<html><head><title>Hello World!! </title></head>\n"; print "<body><h1>Hello world</h1></body></html>\n";
        
        chmod 755 /usr/share/nginx/cgi-bin/hello_world.cgi
      - bash script
        
        /usr/share/nginx/cgi-bin/toto.sh
        
        #!/bin/bash # as PATH is set in /etc/init.d/functions, you may need to complete it # when you are calling other scripts (e.g. in /usr/local/bin) from here PATH=${PATH}:/usr/local/bin export $PATH # call a script in /usr/local/bin my_script.sh # content type must be present, followed by an empty line# all lines must be ended with \r\n (CR LF) echo -e "Content-type: text/plain\r" echo -e "\r" # your text echo "123"
        echo "server software: $SERVER_SOFTWARE" echo "script filename: $SCRIPT_FILENAME"
        echo "toto param: $TOTO_PARAM"
        # print all available environment variables env
        
        chmod 755 /usr/share/nginx/cgi-bin/toto.sh
        
        Debug
        
        /usr/share/nginx/cgi-bin/toto_debug.sh
        
        #!/bin/bash -x # adding -x will make bash xtrace output (by default written to stderr) to appear in /var/log/nginx/error.log, # if option -f was specified in spawn config echo -e "Content-type: text/plain\n" # log to stderr instead of response body (1>&2 echo "[hello]")
        
        ...
        
        Parse GET and POST parameters (using cgi_functions.sh)
        
        See also CGI bash examples with POST
        
        /usr/share/nginx/cgi-bin/toto_with_params
        
        #!/bin/bash# if cgi_functions.sh is installed in /usr/local/bin
        PATH=/usr/local/bin:${PATH} export PATH=$PATH# register all GET and POST variables source cgi_functions.sh cgi_getvars BOTH ALL 1>&2
        # content type must be present, followed by an empty line # all lines must be ended with \r\n (CR LF) echo -e "Content-type: text/plain\r" echo -e "\r" # variables from QUERY and POST are available as shell variables echo "var1: $var1" echo "var2: $var2" echo "var3: $var3"
        
        GET
        
        curl -X GET http://127.0.0.1/cgi-bin/toto_with_params?var1=value1
        
        POST
        
        application/x-www-form-urlencoded
        
        curl -X POST--data 'var3=value3'http://127.0.0.1/cgi-bin/toto_with_params
        
        multipart/form-data
        
        curl -X POST -F var2=value2http://127.0.0.1/cgi-bin/toto_with_params?var1=value1
        
        curl -X POST -F var2=value2 http://127.0.0.1/cgi-bin/toto_with_params
        
        from local:
        
        REQUEST_METHOD=POST QUERY_STRING="var1=value1&var4=value4" var2=value2 var3=value3 /usr/share/nginx/cgi-bin/toto_with_params
        
        echo "var2=value_a&var2=value_b&var3=value3" | (REQUEST_METHOD=POST CONTENT_TYPE=application/x-www-form-urlencoded CONTENT_LENGTH=1000 ./multi_cgi.sh)
        
        Allow only POST:
        
        /usr/share/nginx/cgi-bin/toto_only_post
        
        #!/bin/bash# register all GET and POST variables source cgi_functions.sh cgi_getvars BOTH ALL 1>&2 if [[ $REQUEST_METHOD != "POST" ]] then echo "Status: 405 Method Not Allowed" echo -e "Content-type: text/plain\r" echo -e "\r" echo "Method not allowed" exit 0 fi# content type must be present, followed by an empty line# all lines must be ended with \r\n (CR LF) echo -e "Content-type: text/plain\r" echo -e "\r" ...
    - Client
      - http://127.0.0.1/cgi-bin/hello_world.cgi
      - http://127.0.0.1/cgi-bin/toto.sh
    - Problemes / Problems
      - SELinux
      - 403 Forbidden
        
        chmod +x toto.sh
      - 502 Bad gateway
        
        check that script returns as first lines (optionally a Status line, containing return code number and reason phrase, see: ngx_http_fastcgi_hide_headers) (note the empty line):
        
        Content-type: text/plain
        
        Status: 202 OK Content-type: text/plain
  - Example to execute a fixed script when accessing a url:
    - nginx - How to run a shell script on every request?
    - ...conf with standard parameters
      - location /do_something/ { # Disable gzip (it makes scripts feel slower since they have to complete # before getting gzipped) gzip off; # Set the root to /usr/lib (inside this location this means that we are # giving access to the files under /usr/lib/cgi-bin) root /usr/share/nginx; # Fastcgi socket fastcgi_pass unix:/var/run/fcgiwrap.socket; # Fastcgi parameters, include the standard ones include /etc/nginx/fastcgi_params; # Adjust non standard parameters (SCRIPT_FILENAME) fastcgi_param SCRIPT_FILENAME /path/to/fixed_script.sh; # user-defined parameters fastcgi_param TOTO_PARAM $request_length;
        }
    - fixed_script.sh (see also simple_cgi.sh for an example with POST)
      - #!/bin/bash # content type must be present, followed by an empty line# all lines must be ended with \r\n (CR LF) echo -e "Content-type: text/plain\r" echo -e "\r" # execute some script result=$(/path/to/do_something_script.sh) echo "result:" echo $result # your text echo "request method: $REQUEST_METHOD"
        echo "server software: $SERVER_SOFTWARE" echo "script filename: $SCRIPT_FILENAME"
        echo "toto param: $TOTO_PARAM"
    - fixed_script_with_specified http response code
      - FastCGI and Nginx - Return HTTP Status
      - #!/bin/bash echo "Status: 404 Not Found" echo -e "Content-type: text/plain\r" echo -e "\r" echo "Uep, com anam?"
    - conf with parameters from url
      - location /fast {
            # Disable gzip (it makes scripts feel slower since they have to complete
            # before getting gzipped
            gzip off;
        
            # script will be under
            root /usr/share/nginx;
        
            # Fastcgi socket
            fastcgi_pass unix:/var/run/fcgiwrap.socket;
            # Fastcgi parameters, include the standard ones
            include /etc/nginx/fastcgi_params;
        
            # script to be executed
            fastcgi_param SCRIPT_FILENAME /usr/share/nginx/cgi-bin/fixed_script.sh;
        
            # parameters for the script
            fastcgi_param REQUEST_URI $request_uri;
        
            # parameters from url
            # /fast/<param1>/<param2>/
            if ($request_uri ~* "/fast/([^/]*)/([^/]*)/$" ) {
                set $param1 $1;
                set $param2 $2;
            }
            fastcgi_param PARAM1 $param1;
            fastcgi_param PARAM2 $param2;
        }
    - fixed_script with parameters
      - #!/bin/bash # content type must be present, followed by an empty line # all lines must be ended with \r\n (CR LF) echo -e "Content-type: text/plain\r" echo -e "\r" echo "REQUEST_URI: ${REQUEST_URI}" echo "PARAM1: ${PARAM1}" echo "PARAM2: ${PARAM2}" exit 0
  - Example to get upload time:
    - .../upload.conf
      - location /up { # allow 100 Continue client_max_body_size 20M; # allow upload sendfile on; # Disable gzip (it makes scripts feel slower since they have to complete # before getting gzipped) gzip off; # Set the root to /usr/lib (inside this location this means that we are # giving access to the files under /usr/lib/cgi-bin) root /usr/share/nginx; # Fastcgi socket fastcgi_pass unix:/var/run/fcgiwrap.socket; # Fastcgi parameters, include the standard ones include /etc/nginx/fastcgi_params; # Adjust non standard parameters: fastcgi_param SCRIPT_FILENAME /path/to/upload_time.sh; # user-defined parameters: fastcgi_param REQUEST_LENGTH $request_length; fastcgi_param REQUEST_TIME $request_time; # user-defined headers add_header x-temps $request_time; add_header x-longitud $request_length; }
    - upload_time.sh
      - #!/bin/bash # content type must be present, followed by an empty line# all lines must be ended with \r\n (CR LF) echo -e "Content-type: text/plain\r" echo -e "\r" # execute some script result=$(/path/to/do_something_script.sh) echo "result:" echo $result echo "request method: $REQUEST_METHOD" echo "request length: $REQUEST_LENGTH" echo "request time: $REQUEST_TIME"
    - Client
      - dd if=/dev/zero of=megabyte.dat bs=1M count=1
      - curl -i --data-binary '@megabyte.dat' http://<server>/up

Cache
- ngx_http_headers_module
- cache for hls in nginx-rtmp module
- no-cache for 404 responses (see also cache control for error responses in AWS CloudFront)
  - your.conf
    - server { ... error_page 404 /404.html; location = /404.html { root html; add_header Cache-Control "no-cache" always; add_header Cache-Control "max-age=0" always; } }
CORS
- CORS on Nginx
  - OPTIONS
- How do I add Access-Control-Allow-Origin in NGINX?
- CORS with nginx for 401,404,501 and any other http status
  - add_header 'Access-Control-Allow-Origin' '*' always;

Upload

Test upload speed
- How to measure nginx client's download speed rate
  - $request_length divided by $request_time
Nginx direct file upload without passing them through backend
nginx upload client_max_body_size issue
- Expect: 100-Continue
client
- curl ...
client_body_in_file_only
- files uploaded to
  - /tmp/...
- ...conf
  - server { listen 80; server_name localhost; error_log /var/log/nginx/error.debug.log debug; #sendfile on; location /up { #auth_basic "Restricted Upload"; #auth_basic_user_file basic.htpasswd; #limit_except POST { deny all; } #client_body_temp_path /tmp/; #client_body_in_file_only on; #client_body_buffer_size 128K; # allow 100 Continue client_max_body_size 20M;
    #proxy_pass_request_headers on; #proxy_set_header X-FILE $request_body_file; #proxy_set_body off; #proxy_redirect off; #proxy_pass http://www.example.org/; sendfile on; add_header x-peticio $request; add_header x-temps $request_time; add_header x-longitud $request_length; #root html; #return 200 '$request_body_file'; return 200 'tot bé\n'; }
- Client (POST)
  - curl -i --data-binary '@myfile.png' http://192.168.1.29/upload
- Problemes / Problems
  - FCGIWrap

nginx-upload-module

Upload
Resumable upload
- Resumable uploads over HTTP. Protocol specification
- upload_resumable on
- Usage
  - Extremely Large File Uploads with nginx, Passenger, Rails, and jQuery
nginx-upload-module 2.2 (github)
Nginx upload module (v 2.2.0)
- nginx.service
  - [Service] ... # create hashed dirs for upload-module (level1=1) ExecStartPre=/bin/sh -c '/bin/mkdir -p /var/www/uploads/{0..9}; chmod 777 -R /var/www/uploads/{0..9}'
- Example configuration
- /usr/local/bin/new_upload.sh
  - #!/bin/bash echo -e "Content-type: text/plain\n" source cgi_functions.sh # register all GET and POST variables cgi_getvars BOTH ALL echo "New upload ===============================" export
Other examples
- Uploading to nginx using the nginx upload module with php_handler
  - $_POST[]
Compilation

	client	response code
POST multipart/form-data	`curl -i -X POST -H 'Content-Type: multipart/form-data' -F toto='@myfile.png' http://<server>/up`	200 OK
POST x-www-form-urlencoded	`curl -i --data-binary '@myfile.png' http://<server>/up`	415 Unsupported Media Type
PUT	`curl --upload-file myfile.png http://<server>/up`	405 Not Allowed

Resources (.com)
- NGINX content caching
- Logging and monitoring
i18n
- ...
3rd party modules
- NGINX-based Media Streaming Server (nginx-rtmp-module)
Problemes / Problemes
- check content of:
  - /var/log/nginx/error.log
  - /usr/local/nginx/logs/error.log
- ...
- related to fcgiwrap
  - connect() to unix:/var/run/fcgiwrap.socket failed (11: Resource temporarily unavailable) while connecting to upstream, client: ..., server: ..., request: ..., upstream: "fastcgi://unix:/var/run/fcgiwrap.socket:", host: ...
    - uwsgi with nginx (11: Resource temporarily unavailable) while connecting to upstream)
    - if, for some reason, a script called by fastcgi get stuck, next requests are not being served (netstats reports: CONNECTING), and they are queued.
    - This can be checked with: netstat --unix | grep fcgi | wc
    - is there a limit?
    - workaroud: systemctl restart spawn-fcgi.service
    - to get the process that is blocking fastcgi
    - debug:
      - netstat --unix
      - ss --unix
      - strace
      - lsof | grep fcgi
- related to uWSGI (WSGI)
  - connect() to unix:///var/lib/uwsgi/....sock failed (11: Resource temporarily unavailable) while connecting to upstream
  - connection to page gives "502 Bad Gateway"
    - systemctl status emperor.uwsgi.service
      - import memcache ImportError: No module named memcache
  - connection to page gives "502 Bad Gateway"
    - /var/log/nginx/error.log
      - connect() to unix:///etc/uwsgi/wct_backend.sock failed (13: Permission denied) while connecting to upstream ...
    - If it doesn't work
    - Solució / Solution
      1. regenerate nginx.pp file
      2. sudo systemctl restart emperor.uwsgi.service
      3. sudo systemctl restart nginx.service
  - connection to page gives "502 Bad Gateway"
    - /var/log/nginx/error.log
      - connect() to unix:///etc/uwsgi/wct_backend.sock failed (2: No such file or directory) while connecting to upstream ...
    - Solució / Solution
      - check output from (maybe some misspelling?):
        
        sudo systemctl status emperor.uwsgi.service
      - check that uwsgi is installed in your virtualenv
      - check uwsgi configuration
- related to Django and Daphne (ASGI)
  - connect() failed (111: Connection refused) while connecting to upstream, client: ..., server: ..., request: ..., upstream: "http://[::1]:9000/ws/...", host: ...
    - IPv6 not activated
    - Solució / Solution
      - conf.d/my_nginx.conf
        
        upstream my-upstream { # use 127.0.0.1 instead of localhost to avoid errors related to ipv6 server 127.0.0.1:9000; }
  - [alert] 2398#0: 1024 worker_connections are not enough
    - Optimal value for Nginx worker_connections
    - monitor the number of used worker_connections:
      - sudo ss -tupn | grep nginx | wc -l
      - (deprecated) sudo netstat -tupn | grep nginx | wc -l
    - Solució / Solution
      - nginx.conf
        
        events { # this value must not exceed: # - max number of open files for nginx user and nginx.service: # sudo su - nginx -s /bin/bash -c "ulimit -Hn" # grep LimitNOFILE /usr/lib/systemd/system/nginx.service # - total range of sockets to enable per IP: sysctl net.ipv4.ip_local_port_range # default value: worker_connections 1024; worker_connections 16384; }
      - set max number of open files
      - set total range of sockets to enable per IP
        
        sysctl net.ipv4.ip_local_port_range
        
        sysctl -w net.ipv4.ip_local_port_range ...
  - [error] 22590#0: *344 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: ..., server: ..., request: "GET ...", upstream: "http://127.0.0.1:9000/...", host: ...
    - it seems to be normal
    - Nginx error message when closing websocket connections #1021
  - [info] 22590#0: *355 peer closed connection in SSL handshake while SSL handshaking, client: ..., server: 0.0.0.0:443
    - ...
- accept4() failed (24: Too many open files)
  - Nginx: 24: Too Many Open Files Error And Solution
  - nginx 24: Too many open files
  - see also: worker_connections are not enough
  - Solució / Solution:
    - increase limits for nginx user:
      - present values:
        
        sudo su - nginx -s /bin/bash -c "ulimit -Hn"
        
        sudo su - nginx -s /bin/bash -c "ulimit -Sn"
      - set new values at os level for user nginx:
        
        /etc/security/limits.d/nginx.conf
        
        # default: nginx hard nofile 4096 nginx hard nofile 16384 # default: nginx soft nofile 1024 nginx soft nofile 4096
      - set new values at service level:
        
        /usr/lib/systemd/system/nginx.service
        
        [Service] ... LimitNOFILE=16384
        
        systemctl daemon-reload
        
        systemctl restart nginx.service
      - check new values in running process:
        
        nginx_pid=$(pgrep -f "nginx: master process")
        
        cat /proc/${nginx_pid}/limits | grep -E "Limit|Max open files"
        
        Limit Soft Limit Hard Limit Units Max open files 16384 16384 files
    - perform stress tests
      - monitor active connections
        
        Monitoring Nginx
        
        Module ngx_http_stub_status_module
        
        active connections (e.g. using websockets)
        
        ss -tn src :80 or src :443
        
        netstat -pna | grep :443 | grep ESTABLISHED
        
        ...
      - k6
        
        ...
      - Artillery
        
        Benchmarking websocket server performance with Artillery
        
        Instal·lació / Installation
        
        install nodejs npm
        
        npm install -g artillery
        
        Ús / Usage
        
        artillery run toto.yml
      - ...
- exec: fork failed (12: Cannot allocate memory), client: ..., server: ...
  - Nginx, unable to fork: Cannot allocate memory
  - check available memory and limits
    - free -h
    - ulimit -a
    - swapon -s
- nginx: [emerg] open() "/etc/nginx/nginx.conf" failed (13: Permission denied)
  - Solució / Solution
    - sudo setsebool -P httpd_read_user_content 1
- nginx: [emerg] bind() to 0.0.0.0:8000 failed (13: Permission denied)
  - Solució / Solution
    - generate nginx.pp file
- nginx: [emerg] bind() to 0.0.0.0:444 failed (13: Permission denied)
  - Solució / Solution
    - add port 444 to allowed ports
      - sudo semanage port -a -t http_port_t -p tcp 444
- content is not cached
  - in some cases, content is not cached. E.g. when origin returns Set-Cookie in the response header (see proxy_cache_valid)
  - to disable processing of some origin response header fields, use proxy_ignore_headers

Apache HTTP Server (Apache Software Foundation)

Documentation

How to configure Apache2 (ntu)
Dynamic Content with CGI (CGI)
Environment Variables in Apache

	variable	bash	Python / Django
HttpRequest	`CONTENT_LENGTH` `CONTENT_TYPE` `HTTP_ACCEPT_ENCODING` `HTTP_ACCEPT_LANGUAGE` `HTTP_HOST` `HTTP_REFERER` `HTTP_USER_AGENT` `QUERY_STRING` `REMOTE_ADDR` `REMOTE_HOST` `REMOTE_USER` `REQUEST_METHOD` `SERVER_NAME` `SERVER_PORT`		Django request and response objects
mod_ssl	`SSL_CLIENT_S_DN` `SSL_CLIENT_S_DN` ...
CGI (nginx fastcgi: /etc/nginx/fastcgi_params)	`fastcgi_param QUERY_STRING $query_string;` `fastcgi_param REQUEST_METHOD $request_method;` `fastcgi_param CONTENT_TYPE $content_type;` `fastcgi_param CONTENT_LENGTH $content_length;` `fastcgi_param SCRIPT_NAME $fastcgi_script_name;` `fastcgi_param REQUEST_URI $request_uri;` `fastcgi_param DOCUMENT_URI $document_uri;` `fastcgi_param DOCUMENT_ROOT $document_root;` `fastcgi_param SERVER_PROTOCOL $server_protocol;` `fastcgi_param REQUEST_SCHEME $scheme;` `fastcgi_param HTTPS $https if_not_empty;` `fastcgi_param GATEWAY_INTERFACE CGI/1.1;` `fastcgi_param SERVER_SOFTWARE nginx/$nginx_version;` `fastcgi_param REMOTE_ADDR $remote_addr;` `fastcgi_param REMOTE_PORT $remote_port;` `fastcgi_param SERVER_ADDR $server_addr;` `fastcgi_param SERVER_PORT $server_port;` `fastcgi_param SERVER_NAME $server_name;`	`$QUERY_STRING` `...`
user defined (mod_env)	`PassEnv ...` `SetEnv VARIABLE toto_value` `UnsetEnv ...`

Apache configuration
- apachectl -V
- Virtual hosts
- Secure server (HTTPS)
  - Problemes / Problems
    - Upload
      - access.log
        
        HTTP 413 (Request Entity Too Large)
      - error.log
        
        request body exceeds maximum size for SSL buffer, could not buffer message body to allow SSL renegotiation to proceed.
      - Request Entity Too Large error while uploading files of more than 128KB over SSL
      - Solució / Solution
        
        htttpd.conf
        
        <Directory ...> SSLRenegBufferSize 10486000 </Directory>
- User authentication
- WebDAV
- CGI
- Variables d'entorn / Environment variables
- i18n
  - Apache MultiViews language negotiation set up (w3c)

MPM - Multi-Processing Modules (2.2) (2.4)

Choosing an MPM (2.4)
How do I select which Apache MPM to use? (sf)

Tipus / Types

	module
threaded	event	tries to solve the "keep-alive problem" (2.4: not experimental any more) (not for SSL)
threaded	worker	several threads/process
non-threaded	prefork	1 thread/process

prefork
- 1 thread/process
worker
- several threads/process
event
- tries to solve the "keep-alive problem" (2.4: not experimental any more) (not for SSL)
itk
peruser

Get the present MPM module
- httpd -V
Mostra els mòduls disponibles en compilació / Show available modules at compilation
- httpd -l

Sessions
- mod_session (2.4)
Rendiment / Performance
- Apache performance tuning (2.2) (2.4)
  - Process creation
    - KeepAliveTimeout
- Càrrega del servidor / Server load
  - mod_status
    - Debian / Ubuntu (Apache 2.4)
      - /etc/apache2/mods-available/status.conf
        
        <IfModule mod_status.c> # Allow server status reports generated by mod_status, # with the URL of http://servername/server-status # Uncomment and change the "192.0.2.0/24" to allow access from other hosts. <Location /server-status> SetHandler server-status #Require local #Require ip 192.0.2.0/24 # only from 188.79.*.* : Require ip 188.79 </Location> # Keep track of extended status information for each request ExtendedStatus On # Determine if mod_status displays the first 63 characters of a request or # the last 63, assuming the request itself is greater than 63 chars. # Default: Off #SeeRequestTail On <IfModule mod_proxy.c> # Show Proxy LoadBalancer status in mod_status ProxyStatus On </IfModule> </IfModule>
      - sudo a2enmod status
    - Old way
      - httpd.conf
        
        <IfModule mod_status.c> <Location /server-status> SetHandler server-status Order deny,allow Deny from all allow from 192.168.1 </Location> ExtendedStatus On </IfModule>
    - get the status and refresh every 10 seconds:
      - http://your_ip/server-status?refresh=10
- MPM parameters
  - Paràmetres / Parameters in httpd.conf
  - Check Apache Httpd MPM Config Limits (script)
  - Optimal values for ServerLimit, MaxClients, MaxRequestsPerChild directives
    - Ganglia
    - munin
  - How to calculate Apache MaxClients value and MySQL memory usage
  - High performance web site (slideshare)
  - Tunear Apache con cabeza
  - Performance Tuning Apache and MySQL for Drupal
  - MaxClientsMaxRequestWorkers = ( Total_Memory - OS_Memory - MySQL_Memory ) / Size_per_Apache_process
    - Size_per_Apache_process:
      - ps -yl -C httpd --sort:rss
      - ps -yl -C apache2 --sort:rss
    - OS_Memory:
      - stop Apache and MySQL and then:
        
        ps aux | awk '{sum1 +=$4}; END {print sum1}'
  - Combinació amb un servidor MySQL / MariaDB / Combination with MySQL / MariaDB server
    - MySQL: max_connections > Apache: MaxClientsMaxRequestWorkers
parsed HTML (.shtml)
TkApache
Quick reference card
User Authentication
Authentication (Apache)
- mod_ssl (https config)
  - Environment variables
    - SSL_CLIENT_S_DN
    - SSL_SERVER_S_DN
    - ...
  - Directives
    - SSLCACertificateFile
    - SSLCertificateFile
    - SSLCipherSuite
    - ...
- mod_auth_certificate (*)
  - urpmi apache-mod_auth_certificate
  - /usr/share/doc/apache-mod_auth_certificate/README
  - /etc/httpd/conf/httpd.conf:
    - LoadModule auth_certificate_module /usr/lib/apache-extramodules/mod_auth_certificate.so
Apache::Clean (mod_perl Developer's Cookbook)
Introduction to Server Side Includes
Rewrite
- mod_rewrite
Ten Things You Didn't Know Apache (2.2) Could Do
- Server Name Indicator (SNI)
- mod_substitute
- graceful stop
- mod_proxy_balancer (balanceig de càrrega / load balancing)
- httpd -M
- httx2dbm (rewrite)
- PCRE zero-width assertions
Apache Mobile Filter (AMF)
Modern Mobile Redirect Using .htaccess
- RewriteCond %{REQUEST_URI} !^/mobile/.*$ RewriteCond %{HTTP_USER_AGENT} "android|blackberry|ipad|iphone|ipod|iemobile|opera mobile|palmos|webos|googlebot-mobile" [NC] RewriteRule ^(.*)$ /mobile/ [L,R=302]
Detect Mobile Browser

Fake http server with netcat
Cherokee Web Server (wp)
Netscape Enterprise Server
- Controlling Access to Your Server (.htaccess)
NCSA HTTPd
CERN httpd
Servidors lleugers / Light servers
- LightTPD
- micro_httpd (used in routers)
Jetty Java HTTP Servlet Server
- Views from Abroad: Close Up On Jetty
Web Servers Comparison
Creating Net Sites
Designing Your Web Site
Databases interfaces
Utilitats / Utilities
- WWW FAQs: How do I include one HTML file in another?
- Server Side Scripts (B. Jellinek)
- Web Developer's Virtual Library
- Counter
- Image map
- Search
  - Excite (IRIX)
  - WebGlimpse
- Site map
  - Inxight
- Estadístiques / Statistics
- Validation
  - HTTP validation
  - Nibbler
- Http load testing and benchmarking
  - Howto: Performance Benchmarks a Webserver
  - Stress-testing httperf, siege, apache benchmark, and pronk
  - Apache Benchmark (ab)
    - Using Apache Bench for Simple Load Testing
    - ab – Apache Bench, understanding and getting tangible results.
    - Install
      - Mageia
        
        urpmi apache
      - Ubuntu
        
        apt-get install apache2-utils
      - CentOS
        
        yum install httpd-tools
    - [/usr/sbin/]ab -n number_connections -c concurrent_connections http://example.com/
  - Siege
    - urpmi ...
    - siege ...
  - httperf (HP)
- Wusage Home Page (usage statistics)
- WebTools
- freeWAIS-sf (database)
- Webcal (www calendar)
- webMethods - Automate the Web

Gateway interface

config		gateway interface	backend server
apache	nginx
	proxying to	CGI (Common Gateway Interface)	FastCGI
		WSGI (Python Web Server Gateway Interface)	uwsgi Werkzeug Gunicorn Unit
		ASGI (Asynchronous Server Gateway Interface)	Daphne Uvicorn Unit

CGI (Common Gateway Interface)
- CGI on Apache
- FastCGI
- RFC 3875 "The Common Gateway Interface (CGI) Version 1.1"
  - 4.1 Request Meta-Variables
    - QUERY_STRING
    - REMOTE_ADDR
    - REMOTE_PORT
    - ...
- The Web Developer's Virtual Library course
- CGIHTML (C routines)
- CGI Developer's Guide
- Bash
  - Bash.CGI
    - cgi_functions.sh
      - #!/bin/bash # https://oinkzwurgl.org/hacking/bash_cgi/ # Phillippe Kehi <phkehi@gmx.net> and flipflip industries # to test this function locally: # source cgi_functions.sh # QUERY_STRING="var1=value1" cgi_getvars GET var1; echo $var1 # QUERY_STRING="var1=value1&var2=value2" cgi_getvars GET ALL; echo $var1; echo $var2 # CONTENT_TYPE="application/x-www-form-urlencoded" ... cgi_getvars POST ALL; echo $var1; echo $var2 # to test this function locally: # source cgi_functions.sh # QUERY_STRING="var1=value1" cgi_getvars GET var1; echo $var1 # QUERY_STRING="var1=value1&var2=value2" cgi_getvars GET ALL; echo $var1; echo $var2 # CONTENT_TYPE="application/x-www-form-urlencoded" ... cgi_getvars POST ALL; echo $var1; echo $var2 # (internal) routine to store POST data function cgi_get_POST_vars() { # check content type # FIXME: not sure if we could handle uploads with this.. if [ "${CONTENT_TYPE}" != "application/x-www-form-urlencoded" ] && ! [[ "${CONTENT_TYPE}" =~ ^"multipart/form-data" ]] then # if, in /etc/sysconfig/spawn-fcgi, OPTIONS contains -f, # this echo can be seen at /var/log/nginx/error.log echo "[`basename $0`] WARNING: received CONTENT_TYPE is ${CONTENT_TYPE}, but implemented content types for POST are only: "\ "application/x-www-form-urlencoded, multipart/form-data" 1>&2 return fi # save POST variables (only first time this is called) [ -z "$QUERY_STRING_POST" \ -a "$REQUEST_METHOD" = "POST" -a ! -z "$CONTENT_LENGTH" ] && \ read -N $CONTENT_LENGTH RECEIVED_POST #echo "RECEIVED_POST: $RECEIVED_POST" if [[ "${CONTENT_TYPE}" =~ ^"multipart/form-data" ]] then # export variables from multipart boundary=$(echo $CONTENT_TYPE | awk -F= '{print$2}'); #QUERY_STRING_POST=$(echo "$RECEIVED_POST" | awk -v b=$boundary 'BEGIN {RS=b"\r\n";FS="\r\n";ORS="&"} $1 ~ /^Content-Disposition/ {gsub(/Content-Disposition: form-data; name=/,"",$1); gsub("\"","",$1); print $1"="$3}') # variables must be exported here, because they can contain ampersands in value eval $(echo "$RECEIVED_POST" | awk -v b=$boundary 'BEGIN {RS=b"\r\n";FS="\r\n";ORS=" "} $1 ~ /^Content-Disposition/ {gsub(/Content-Disposition: form-data; name=/,"",$1); gsub("\"","",$1); print "export "$1"=\""$3"\""}') else # take input string as is QUERY_STRING_POST="$RECEIVED_POST" fi return }
        
        # (internal) routine to decode urlencoded strings function cgi_decodevar() { [ $# -ne 1 ] && return local v t h # replace all + with whitespace and append %% t="${1//+/ }%%" while [ ${#t} -gt 0 -a "${t}" != "%" ]; do v="${v}${t%%\%*}" # digest up to the first % t="${t#*%}" # remove digested part # decode if there is anything to decode and if not at end of string if [ ${#t} -gt 0 -a "${t}" != "%" ]; then h=${t:0:2} # save first two chars t="${t:2}" # remove these v="${v}"`echo -e \\\\x${h}` # convert hex to special char fi done # return decoded string echo "${v}" return } # routine to get variables from http requests # usage: cgi_getvars method varname1 [.. varnameN] # method is either GET or POST or BOTH # the magic varible name ALL gets everything function cgi_getvars() { [ $# -lt 2 ] && return local q p k v s # get query case $1 in GET) [ ! -z "${QUERY_STRING}" ] && q="${QUERY_STRING}&" ;; POST) cgi_get_POST_vars [ ! -z "${QUERY_STRING_POST}" ] && q="${QUERY_STRING_POST}&" ;; BOTH) [ ! -z "${QUERY_STRING}" ] && q="${QUERY_STRING}&" cgi_get_POST_vars [ ! -z "${QUERY_STRING_POST}" ] && q="${q}${QUERY_STRING_POST}&" ;; esac shift s=" $* " # parse the query data while [ ! -z "$q" ]; do p="${q%%&*}" # get first part of query string k="${p%%=*}" # get the key (variable name) from it v="${p#*=}" # get the value from it q="${q#"$p"&*}" # strip first part from query string # decode and assign variable if requested [ -n "$p" ] && [ "$1" = "ALL" -o "${s/ $k /}" != "$s" ] && \ export "$k"="`cgi_decodevar \"$v\"`" done return }
  - CGI shell script using bash
  - Bash
    - Saving a binary file
  - How to parse $QUERY_STRING from a bash CGI script
  - Creating CGI Programs with Bash: Handling POST Data
    - curl -i -X POST --data 'tata=tata_value' http://127.0.0.1/test
  - Examples with POST
    - See also Parse GET and POST parameters
    - simple_cgi.sh
      - #!/bin/bash code="200 OK" echo -e "Status: ${code}\r" echo -e "Content-type: text/plain\r" echo -e '\r' echo -e "REQUEST_METHOD: ${REQUEST_METHOD}\r" echo -e "CONTENT_TYPE: ${CONTENT_TYPE}\r" echo -e "CONTENT_LENGTH: ${CONTENT_LENGTH}\r" echo -e "QUERY_STRING: ${QUERY_STRING}\r" exit 0
      - #!/bin/bash code="200 OK" echo -e "Status: ${code}\r" echo -e "Content-type: text/plain\r" echo -e 'Access-Control-Allow-Origin:*\r' echo -e 'Access-Control-Allow-Credentials: true\r' echo -e 'Access-Control-Allow-Methods: GET, POST, OPTIONS\r' echo -e 'Access-Control-Allow-Headers: DNT, X-CustomHeader, Keep-Alive, User-Agent, X-Requested-With, If-Modified-Since, Cache-Control, Content-Type, Authorization \r' echo -e '\r' echo -e "REQUEST_METHOD: ${REQUEST_METHOD}\r"echo -e "QUERY_STRING: ${QUERY_STRING}\r" echo -e "CONTENT_TYPE: ${CONTENT_TYPE}\r" echo -e "CONTENT_LENGTH: ${CONTENT_LENGTH}\r" exit 0
      - #!/bin/bash # when called with: # curl -i "http://localhost/?a=b&c=d" --data "toto=tu" --data "toto=ta" # result is: # REQUEST_METHOD: POST # QUERY_STRING: a=b&c=d # CONTENT_TYPE: application/x-www-form-urlencoded # CONTENT_LENGTH: 15 # CONTENT: toto=tu&toto=ta code="200 OK" echo -e "Status: ${code}\r" echo -e "Content-type: text/plain\r" echo -e "REQUEST_METHOD: ${REQUEST_METHOD}\r" echo -e "QUERY_STRING: ${QUERY_STRING}\r" echo -e "CONTENT_TYPE: ${CONTENT_TYPE}\r" echo -e "CONTENT_LENGTH: ${CONTENT_LENGTH}\r" read -N ${CONTENT_LENGTH} CONTENT echo -e "CONTENT: ${CONTENT}\r" exit 0
      - #!/bin/bash -x echo -e "Content-type: text/plain\n" echo "content type: ${CONTENT_TYPE}" echo "QUERY_STRING_POST:" cat
      - #!/bin/bash -x echo -e "Content-type: text/plain\n" echo "content type: ${CONTENT_TYPE}" read -N $CONTENT_LENGTH QUERY_STRING_POST echo "QUERY_STRING_POST:" echo "$QUERY_STRING_POST"
    - parse_multi.sh
      - #!/bin/bash echo -e "Content-type: text/plain\n" echo "CONTENT_TYPE: $CONTENT_TYPE" echo "CONTENT_LENGTH: $CONTENT_LENGTH" # get boundary from CONTENT_TYPE boundary=$(echo $CONTENT_TYPE | awk -F= '{print$2}') echo "== boundary:" echo "$boundary" # read standard input read -N $CONTENT_LENGTH QUERY_STRING_POST echo "== received standard input:" echo "$QUERY_STRING_POST" # parse standard input in multipart/form-data post_params=$(echo "$QUERY_STRING_POST" | awk -v b=$boundary 'BEGIN {RS=b"\r\n";FS="\r\n";ORS="&"} $1 ~ /^Content-Disposition/ {gsub(/Content-Disposition: form-data; name=/,"",$1); gsub("\"","",$1); print $1"="$3}') echo "== post_params:" echo "$post_params" exit 0
- Perl
  - CGI (CPAN)
    - CGI::Simple
- Python
WSGI (Python Web Server Gateway Interface)
- Nginx proxy to WSGI

ASGI (Asynchronous Server Gateway Interface)

Nginx proxy to ASGI
Daphne
- Daphne with Django
- ...
Uvicorn

Unit: Universal Web app server

Can be used with
- FastAPI

Instal·lació / Installation

Nginx integration
- http { upstream unit_backend { server 127.0.0.1:8080; } server { location /unit/ { proxy_pass http://unit_backend/; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } } }
- http { server { location /unit/ { # Note the trailing slash on the proxy_pass. # It tells nginx to replace /unit/ with / when passing the request, including queryparams # if required queryparms were not received by unit, an error 422 would rise proxy_pass http://127.0.0.1:8080/; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } } }

Docker

Unit in Docker
Docker images
- no need to upload config using sockets; just put config.json under /docker-entrypoint.d/
Docker FastAPI projects
- Nginx

	Flask	FastAPI
local: /path/to/app/	webapp/wsgi.py `from flask import Flask app = Flask(__name__) @app.route('/') def hello_world(): return 'Hello, World!'`	webapp/asgi.py `from fastapi import FastAPI` `app = FastAPI()` `@app.get("/") async def root(): return {"message": "Hello, World!"}`
	requirements.txt `flask`	requirements.txt `fastapi`
	config/config.json `{ "listeners":{ "*:8000":{ "pass":"applications/webapp" } }, "applications":{ "webapp":{ "type":"python 3", "path":"/www/", "module": "wsgi", "callable": "app" } } }`	config/config.json `{ "listeners":{ "*:8000":{ "pass":"applications/fastapi" } }, "applications":{ "fastapi":{ "type":"python 3.12", "path":"/www/", "module": "asgi", "callable": "app" } } }`
	log/unit.log	log/unit.log
	state/	state/
	Dockerfile `FROM unit:1.34.2-python3.12 COPY requirements.txt /config/requirements.txt RUN python3 -m pip install -r /config/requirements.txt`	Dockerfile `FROM unit:1.34.2-python3.12 COPY requirements.txt /config/requirements.txt RUN python3 -m pip install -r /config/requirements.txt # CMD ["unitd-debug","--no-daemon","--control","unix:/var/run/control.unit.sock"]`
build docker image	`docker build --tag=unit-webapp .`	`docker build --tag=unit-fastapi .`
run container	`#` port 8080 in host corresponds to port 8000 in container export UNIT=$( \ docker run -d \ --mount type=bind,src="$(pwd)/config/",dst=/docker-entrypoint.d/ \ --mount type=bind,src="$(pwd)/log/unit.log",dst=/var/log/unit.log \ --mount type=bind,src="$(pwd)/state",dst=/var/lib/unit \ --mount type=bind,src="$(pwd)/webapp",dst=/www \ -p 8080:8000 unit-webapp \ )	# port 8080 in host corresponds to port 8000 in container export UNIT=$( \ docker run -d \ --mount type=bind,src="$(pwd)/config/",dst=/docker-entrypoint.d/ \ --mount type=bind,src="$(pwd)/log/unit.log",dst=/var/log/unit.log \ --mount type=bind,src="$(pwd)/state",dst=/var/lib/unit \ --mount type=bind,src="$(pwd)/webapp",dst=/www \ -p 8080:8000 unit-fastapi \ )
test GET	`curl -X GET localhost:8080`
documentation	`http://localhost:8080/docs`
get config	`docker exec -ti $UNIT curl -X GET --unix-socket /var/run/control.unit.sock http://localhost/config/`
restart application	`docker exec -ti $UNIT curl -X GET --unix-socket /var/run/control.unit.sock http://localhost:8000/control/applications/webapp/restart`	`docker exec -ti $UNIT curl -X GET --unix-socket /var/run/control.unit.sock http://localhost:8000/control/applications/fastapi/restart`

Control API
...

Servlets / JSP servers
C Server Pages (CSP)
MIME types configuration

Servidors FTP / FTP servers

Clients FTP
vsftpd
- CentOS
  - FTP
  - 22.5. vsftpd Configuration Options
ProFTPD
- Instal·lació / Installation
  - CentOS
    - sudo yum install proftpd
    - systemctl start proftpd
- Mini-Howto
  - Debugging
    - /usr/lib/systemd/system/proftpd.service
      - ExecStart = /usr/sbin/proftpd -d10 $PROFTPD_OPTIONS
  - Configuring a directory
  - Virtual users
    - Passos / Steps
      - sudo -i
      - cd /etc; mkdir proftpd; cd proftpd
      - wget https://raw.githubusercontent.com/proftpd/proftpd/master/contrib/ftpasswd
      - chmod +x ftpasswd
      - ./ftpasswd --group -gid 9000 --name ftp_users
      - ./ftpasswd --passwd --name user1 --home /home --shell /bin/bash --uid 8000 --gid 9000
      - /etc/proftpd.conf
        
        # users AuthUserFile /etc/proftpd/ftpd.passwd AuthGroupFile /etc/proftpd/ftpd.group
      - systemctl restart proftpd.service
    - Test
      - ftp://user1...
  - AWS and ProFTPD
    - security groups
      - tcp 21
      - tcp 1024 - 1048
    - proftpd.conf
      - # aws PassivePorts 1024 1048 MasqueradeAddress <your_public_ip_address>
- /etc/proftp.conf
  - DefaultRoot ~
  - <Directory ~> <Limit ALL> DenyAll </Limit> <Limit DIRS READ> AllowAll </Limit> </Directory> <Directory ~/upload> <Limit WRITE> AllowAll </Limit> </Directory>

Clients FTP / FTP Clients

sftp
lftp
- download
  - lftpget ftp://user@remote_host/dir1/dir2/dir3/file1
  - lftp ftp://user@remote_host/dir1 -e "get dir2/dir3/file1; bye"
  - mirror
    - lftp ftp://user@remote_host/dir1 -e "mirror; bye"
- upload
  - reverse (-R) mirror
    - lftp -d -f commands.lftp 2>> toto.log
      - commands.lftp
        
        open -u user,password remote_host set ftp:ssl-allow no mirror -R local_dir
  - Exclude .git dir:
    - lftp exclude syntax confusion
    - commands.lftp
      - open -u user,password remote_host set ftp:ssl-allow no mirror --exclude --exclude=^\.git/$ -R local_dir remote_dir
  - How to use rsync over ftp
curlftpfs (mount locally)
- Mount remote ftp directory host locally into linux filesystem
weex (updating web pages)
Python

PHP

Pàgines web dinàmiques / Dynamic web pages
- Frameworks
PEAR packages
PHP CLI (command line)
- PHP CLI
- Using PHP from the command line
- Instal·lació / Installation
  - Mageia
    - urpmi php-cli
- Exemples / Examples: (-r to avoid <?php...?>)
  - php [-f] toto.php
  - php -r "mail('to_user@example.org','subjecte','cos del missatge','From:from_user@example.org' );"
IDE
- kdevelop
- Eclipse
  - PDT
    - Xdebug
PHP debug
- php -i
- /etc/php.ini (Mageia: urpmi php-ini) or /etc/php5/apache2/php.ini (Debian/Ubuntu)
  - display_errors = On
- service httpd restart (or: sudo service apache2 restart)
- comprova la sintaxi / check syntax
  - urpmi php-cli
  - php -l toto.php
- /etc/php.ini or /etc/php5/apache2/php.ini
  - error_reporting = E_ALL
  - html_errors = On
- logs
  - error_log
  - Examples
    - error_log("You messed up!", 3, "/var/tmp/my-errors.log");
    - error_log(print_r($variable, TRUE));
    - error_log("You messed up!", 3, "php://stdout");
- see also Apache logs
- check the output
  - php -f toto.php
  - if result is empty and "echo $?" returns 255, it could mean that dependencies not satisfied
- xdebug
  - Instal·lació / Installation
    - Mageia
      - urpmi php-xdebug
    - Ubuntu
      - sudo apt-get install php5-xdebug
  - Eclipse
    - Debugging using XDebug
    - Setup php ini file:
      - Mageia: /etc/php.d/A29_xdebug.ini
      - Ubuntu: /etc/php5/apache2/conf.d/20-xdebug.ini
      - xdebug.remote_enable = 1 xdebug.remote_handler = 'dbgp'
    - An http server must be configured and running
      - Apache
    - Eclipse: Window / Preferences / PHP / Servers
      - Server
        
        Base URL: http://localhost
        
        Document Root: /var/www/html
      - Debugger: XDebug
        
        Port: 9000
    - Problems
      - Progress tab: "waiting for xdebug session"
        
        Solution
        
        xdebug.remote_handler = 'dbgp'
  - kdevelop
    - KDevelop XDebug (Php Debugger)
- Debugging remote CLI with phpstorm
Dependencies
- inclued
  - Installation
    - Mageia
      - urpmi php-inclued
Errors
- Reference - What does this error mean in PHP?
- "Warning: cannot modify header information..."
  - Si a php.ini hi ha:
    - error_reporting = E_ALL & ~E_DEPRECATED
    - i es produeix un "Notice:...", es posarà al text de resposta, i el codi serà un 200, i ja no es podrà canviar per un altre
  - En canvi, si a php.ini hi ha:
    - error_reporting = E_ALL & ~E_DEPRECATED & ~E_NOTICE
    - no s'escriurà cap "Notice:..." a la resposta, i el codi HTTP es podrà canviar mitjançant un "header("HTTP/1.0 204 No Response")"
  - Penseu també a eliminar les línies en blanc o retorns de carro fora de <php>
i18n
- gettext
  - PHP internationalization with gettext tutorial
  - Localizing PHP Applications “The Right Way”, Part 1
  - PHP internationalization with gettext tutorial
  - WordPress
    - How to Easily Create a Multilingual WordPress Site
    - i18 for WordPress Developers
  - Accept-Language (HTTP request header)
    - How to use Locale::acceptFromHttp without a filter list?
  - dependencies
    - Mageia
      - urpmi php-gettext php-intl
  - Steps
    - cd public
    - create a php file for testing:
      - public/test_locale.php
        
        <?php // get language from browser preferences (Accept-Language http header) // only works for the first option in the browser $lang_from_http_header = Locale::acceptFromHttp($_SERVER['HTTP_ACCEPT_LANGUAGE']); echo "lang_from_http_header: " . $lang_from_http_header . "<br/>"; putenv("LANGUAGE=".$lang_from_http_header); // setup directory and domain (file name: <domain>.mo) $domain = "messages"; bindtextdomain($domain,"Locale"); bind_textdomain_codeset($domain, 'UTF-8'); textdomain($domain); // why is this needed? // this locale has nothing to do with the language // this locale must be installed in the system $locale = "en_US"; setlocale(LC_MESSAGES, $locale); echo _("Hello world!"); //phpinfo(); ?>
    - extract all translatable strings to a portable object template file:
      - xgettext --from-code=UTF-8 -o php.pot *.php
      - find . -iname "*.php" -exec xgettext -a -L PHP --from-code=UTF-8 -j -o php.pot {} \;
    - create dir structure
      - mkdir -p Locale/ca/LC_MESSAGES
      - mkdir -p Locale/fr/LC_MESSAGES
      - ...
    - edit messages.pot
      - "Content-Type: text/plain; charset=UTF-8\n"
    - create po files from pot
      - cp php.pot Locale/ca/LC_MESSAGES/messages.po
    - cd Locale/ca/LC_MESSAGES
    - edit messages.po to update translations
    - compile po to mo:
      - msgfmt -o messages.mo messages.po
    - (?) restart Apache server
      - # systemctl restart httpd
  - Problems
    - PHP Gettext problems (like non-thread-safe?)
      - Solutions
        
        nginx
        
        Apache threaded or non threaded + PHP fastcgi
        
        Apache non threaded + mod-php
- PHP internationalization – i18n mechanisms tutorial
- How to use Locale::acceptFromHttp without a filter list?
- Smarty
Smarty
- i18n
  - smarty-gettext
    - Smarty gettext plugin
    - Installation
      - cd <smarty>/plugins
      - wget https://raw.githubusercontent.com/smarty-gettext/smarty-gettext/master/block.t.php
      - wget https://raw.githubusercontent.com/smarty-gettext/smarty-gettext/master/function.locale.php
    - Setup
      - cd public
      - get script to extract translatable strings
        
        wget https://raw.githubusercontent.com/smarty-gettext/smarty-gettext/master/tsmarty2c.php
        
        chmod +x tsmarty2c.php
      - edit your master tpl file:
        
        {locale path="Locale" domain="messages"}
      - edit your_view.tpl
        
        {t}Hello world from Smarty!{/t}
      - extract all translatable strings to a pot file:
        
        ./tsmarty2c.php -o smarty.pot <path_to>/views/
      - merge pot files from *.php and *.tpl
        
        xgettext --from-code=UTF-8 -o php.pot *.php
        
        msgcat -o messages.pot php.pot smarty.pot
      - create po files from pot
        
        cp messages.pot Locale/ca/LC_MESSAGES/messages.po
      - cd Locale/ca/LC_MESSAGES
      - edit messages.po to update translations
      - compile po to mo:
        
        msgfmt -o messages.mo messages.po
    - Next times:
      - regenerate pot
        
        xgettext --from-code=UTF-8 -o php.pot *.php
        
        ./tsmarty2c.php -o smarty.pot <path_to>/views/
        
        msgcat -o messages.pot php.pot smarty.pot
      - update po
        
        msgmerge -U Locale/ca/LC_MESSAGES/messages.po messages.pot
        
        msgmerge -U Locale/fr/LC_MESSAGES/messages.po messages.pot
        
        ...
      - recompile mo
        
        msgfmt -o Locale/ca/LC_MESSAGES/messages.moLocale/ca/LC_MESSAGES/messages.po
        
        msgfmt -o Locale/fr/LC_MESSAGES/messages.moLocale/fr/LC_MESSAGES/messages.po
        
        ...
Correu electrònic / E-mail
- PHPMailer (supports direct SMTP)
curl-php
- PHP Curl with port number issue
- POST of array to djangorestframework, in multipart (PhpMultiPartParser)
  - <?php // http://stackoverflow.com/questions/3772096/posting-multidimensional-array-with-php-and-curl function http_build_query_for_curl( $arrays, &$new = array(), $prefix = null ) { if ( is_object( $arrays ) ) { $arrays = get_object_vars( $arrays ); } foreach ( $arrays AS $key => $value ) { $k = isset( $prefix ) ? $prefix . '[' . $key . ']' : $key; if ( is_array( $value ) OR is_object( $value ) ) { http_build_query_for_curl( $value, $new, $k ); } else { $new[$k] = $value; } } } $ch = curl_init(); $url = "http://127.0.0.1:8000/mymodels/"; $myvalues = array("first", "second"); $post_fields = array("name" => "toto", "myvalues" => $myvalues ); http_build_query_for_curl( $post_fields, $post ); print_r($post); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: multipart/form-data')); curl_setopt($ch, CURLOPT_POST, sizeof($post_fields)); curl_setopt($ch, CURLOPT_POSTFIELDS, $post ); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); $server_output = curl_exec($ch); $info = curl_getinfo($ch); if(curl_errno($ch)) { echo "something was wrong in your request"; } curl_close($ch); print $server_output; ?>
Certificats / Certificates
- Using SSL client certificates with PHP
- PHP: Cryptography extensions
  - OpenSSL
- Example (see also Apache SSL variables):
  - <?php if ($_SERVER['SSL_CLIENT_VERIFY']=='SUCCESS') { echo "CN="; echo $_SERVER['SSL_CLIENT_S_DN_CN']; $msg = 'OK'; } else { $msg = 'FAIL'; } echo $msg; ?>
Cross-Origin Resource Sharing (CORS)
- Requests with credentials
- Server-side Acess Control (PHP examples)
- http://foo.example/ (Javascript)
  - var invocation = new XMLHttpRequest(); var url = 'https://bar.other/' invocation.onreadystatechange = function() { if (invocation.readyState!=4) return; var is_ok = invocation.responseText=='OK'; if (is_ok) { document.location.href = isok.html } else { document.location.href = isnotok.html
    } } invocation.open('GET', url, true );invocation.withCredentials = "true"; invocation.send();
- https://bar.other/ (PHP)
  - <?php header('Access-Control-Allow-Origin: http://foo.example/'); header('Access-Control-Allow-Credentials: true'); if ($_SERVER['SSL_CLIENT_VERIFY']=='SUCCESS') { echo 'OK'; } else { $sslcn = $_SERVER['SSL_CLIENT_S_DN_CN']; $msg = "$sslcn NOT_OK"; echo$msg; } ?>
- Nota: si no es fan servir Access-Control-Allow- (o bé "Access-Control-Allow-Origin: *"), no estarà disponible responseText a la URL origen. (Firefox: Cross-domain requests with credentials return empty)
- "Access-Control-Allow-Origin: *" no pot estar lligat a withCredentials = "true"
  - Solució / Solution
    - Access-Control-Allow-Origin: http...
      Access-Control-Allow-Credentials: true
HTML parsing
PHP3 (es) (ch)
Databases interaction
Emacs modes
- Php Mode (EmacsWiki)
  1. cd /usr/share/emacs/24.2/lisp/progmodes/
  2. wget http://php-mode.svn.sourceforge.net/svnroot/php-mode/tags/php-mode-1.5.0/php-mode.el
  3. ~/.emacs
LBD/EPFL
Manual (local)
Tutorial
PHP Nuke
Suhosin (Hardened-PHP project - PHP security)

Crawling

Scrapy

Robots

Robots exclusion

Seguretat / Security

Criptografia / Cryptography
Seguretat en informàtica / Computer security
Programari / Software
- Programari de seguretat / Security software
- Programari de criptografia / Cryptography software
HTTPS
- Apache HTTPS server
The World Wide Web Security FAQ
OpenSSL
SSLeay
Applications with SSL
Check
- Cryptozilla
- Fortify SSL Encryption Report
- Certifite Test Page (sign)
The Twenty Most Critical Internet Security Vulnerabilities

Autenticació i autorització d'usuari / User authentication and authorisation

authentication protocol

uses authorisation
framework

provider

document served by provider

client application

mechanism

Kerberos

SAML

SAML identity provider (IdP)

Auth0
Okta
OneLogin	IdP Metadata
PingIdentity
FreeIPA

assertion (signed XML):

subject information
attributes
issuer
other info about authentication event

SAML service provider (SP) (always a web site)

SP Metadata

the user (principal) is redirected from SP to IDP for sign in

OpenID Connect (OIDC)

OAuth 2.0

OpenID Provider (OP)

Facebook
Google
...

id_token (signed JSON)

subject
issuer
authentication information

Relaying Party (RP) (a web or mobile application)

URL Callback

the user is redirected from RP to OP for sign in

Smart card

Info
- The Sun BabelFish Blog
- OAuth vs. SAML vs. OpenID Connect
  - Front channel vs back channel
- SAML vs. OAuth: Comparison and Differences

Standards

Authentication protocols (l'usuari demostra, via un tercer, que és qui diu que és)
- OpenID
  - OpenID (wp)
    - OpenID versus pseudo-authentication using OAuth
- SAML
  - SAML (wp)
  - SAML for Web Developers
    - hand-drawn pictures
  - Implementacions / Implementations
    - SAML2 in Django
  - Identity Management platforms (List of single sign-on implementations)
    - Google
      - Set up your own custom SAML app
    - FreeIPA
      - Installation
        
        Docker (freeipa/freeipa-container) (dockerhub)
        
        verify the cgroup version used in your computer
        
        grep cgroup /proc/filesystems
        
        v1:
        
        nodev   cgroup
        
        v2:
        
        nodev   cgroup nodev   cgroup2
        
        if you have cgroup v2:
        
        /etc/docker/daemon.json
        
        { "userns-remap": "default" }
        
        not working:
        
        {
        "features": {
            "buildkit": true
        },
        "userns-remap": "default"
        }
        
        sudo systemctl restart docker.service
        
        cd
        
        mkdir ipa-data
        
        docker run --sysctl net.ipv6.conf.all.disable_ipv6=0 --name freeipa-server-container -ti -h ipa.example.test --read-only -v $(pwd)/ipa-data:/data:Z freeipa/freeipa-server:almalinux-9

Authorisation frameworks (l'usuari dóna permís a un tercer per a accedir a un servidor on està registrat, però sense passar-li les credencials)

OAuth - Open Authorization (wp)

l'usuari (resource owner) permet l'accés a un servidor (resource server) per part d'una altra aplicació (third-party app, client), a qui l'usuari ha autoritzat (via authorization server), però sense passar-li les credencials

exemples de serveis que ens permeten crear un compte de desenvolupador i crear aplicacions, que es publicaran (o no; per a ús personal) i permetran a altres usuaris utilitzar el servei mitjançant les nostres aplicacions

quan creeu l'aplicació us demanarà:
- Application ID: ...
- OAuth Flow: authorization code, ....
- Redirect URI: REDIRECT_URI
i us proporcionarà:
- CLIENT_ID
- CLIENT_SECRET

servei	dev program	...
Facebook
Google
Home Connect	Home Connect Developer Program
LinkedIn
Twitter
...

OpenID vs. pseudo-authentication using OAuth
OAuth v1
- RFC 5849 (1.0)
- OAuth Core 1.0
  - 6. Authenticating with OAuth

OAuth 2

OAuth 2 Simplified (for developers)
OAuth 2.0 Framework: RFC 6749
OAuth 2.0 Bearer Tokens - RFC 6750

Grant types (OAuth 2 Simplified)

...

grant type
authorization code	web server apps (served by your server)	Your server uses client secret (`CLIENT_SECRET`) when communicating with the authorization server, because apps run on server. Authorization create a login link which sends the user to the authorization server: `https://authorization-server.com/auth? response_type=code& client_id=CLIENT_ID& redirect_uri=REDIRECT_URI& scope=photos& state=1234zyx` `response_type=code` your server expects an authorization code `client_id=...` the client id for your app `redirect_uri=` the URI to return the user to after authorization is complete `scope=...` which parts of the user's account you wish to access `state=...` a random string generated by your application a prompt generated by authorization server appears to the user, where the user can deny or allow. the user is redirected to your app server and information is carried by queryparams: `https://example-app.com/cb? code=AUTH_CODE_HERE& state=1234zyx` Getting an access token: your server connects to authorization server to get an access token: `POST https://api.authorization-server.com/token? grant_type=authorization_code& code=AUTH_CODE_HERE& redirect_uri=REDIRECT_URI& client_id=CLIENT_ID& client_secret=CLIENT_SECRET` the authorization server replies `{ "access_token":"RsT5OjbzRn430zqMLgV3Ia", "expires_in":3600 }` your server connects the resource server using the access token
	browse-based (single-page apps)	"App runs entirely in the browser after loading the source code from a web page. Since the entire source code is available to the browser, they cannot maintain the confidentiality of a client secret, so the secret is not used in this case. The flow is based on the authorization code flow above, but with the addition of a dynamically generated secret used on each request. This is known as the PKCE (proof key for code exchange) extension" (`CODE_VERIFIER`, `CODE_CHALLENGE`). Authorization create (e.g. using PKCE tools): a random string, 43-128 characters long (`CODE_VERIFIER`) hash it with SHA256, then code it in base64 (`CODE_CHALLENGE`) create a login link which sends the user to the authorization server: `https://authorization-server.com/auth? response_type=code& client_id=CLIENT_ID& redirect_uri=REDIRECT_URI& scope=photos& state=1234zyx& code_challenge=CODE_CHALLENGE& code_challenge_method=S256` a prompt generated by authorization server appears to the user, where the user can deny or allow. the user is redirected to your app server and information is carried by queryparams: `https://example-app.com/cb? code=AUTH_CODE_HERE& state=1234zyx` Getting an access token: el vostre servidor connecta amb el servidor d'autorització per a obtenir un token `POST https://api.authorization-server.com/token? grant_type=authorization_code& code=AUTH_CODE_HERE& redirect_uri=REDIRECT_URI& client_id=CLIENT_ID& code_verifier=CODE_VERIFIER` el servidor respon: `{ "access_token":"RsT5OjbzRn430zqMLgV3Ia", "expires_in":3600 }` your server connects the resource server using the access token
	mobile apps	Authorization create a log-in button Using the service's native app (e.g. Facebook app when installed) `fbauth2://authorize? response_type=code& client_id=CLIENT_ID& redirect_uri=REDIRECT_URI& scope=email& state=1234zyx` `code_challenge=CODE_CHALLENGE& code_challenge_method=S256` Using a web browser `https://facebook.com/dialog/oauth? response_type=code& client_id=CLIENT_ID& redirect_uri=REDIRECT_URI& scope=email& state=1234zyx` a prompt appears to the user, where the user can deny or confirm the user is redirected to your app with an url like: `fb00000000://authorize? code=AUTH_CODE_HERE& state=1234zyx` Getting an access token `POST https://api.authorization-server.com/token grant_type=authorization_code& code=AUTH_CODE_HERE& redirect_uri=REDIRECT_URI& client_id=CLIENT_ID& code_verifier=CODE_VERIFIER` el servidor respon: `{ "access_token":"RsT5OjbzRn430zqMLgV3Ia", "expires_in":3600 }` your server connects the resource server using the access token
password	username and password (only first-party apps)	"Since this obviously requires the application to collect the user's password, it must only be used by apps created by the service itself. For example, the native Twitter app could use this grant type to log in on mobile or desktop apps." Getting an access token `POST https://api.authorization-server.com/token grant_type=password& username=USERNAME& password=PASSWORD& client_id=CLIENT_ID`
client credentials	application access without a user present	"In some cases, applications may need an access token to act on behalf of themselves rather than a user. For example, the service may provide a way for the application to update their own information such as their website URL or icon, or they may wish to get statistics about the users of the app." Getting an access token `POST https://api.authorization-server.com/token grant_type=client_credentials& client_id=CLIENT_ID& client_secret=CLIENT_SECRET`
implicit

...
Implementations
- Server
  - ...
- Client
  - Python
    - rauth

Single sign-on (SSO)
- Info
  - (wp)
  - What Is Single Sign-On (SSO)?
- Autenticació social / Social authentication
  - Info
    - Social login (wp)
  - Clients
    - Autenticació social amb Django / Social authentication in Django
  - Proveïdors
    - Facebook Login
      - Access tokens
      - Graph API (OAuth 2)
      - Manually Build a Login Flow
      - Creation and setup of a Facebook app:
        
        create an application at https://developers.facebook.com/
        
        MyApps -> Add a new app
        
        configure the Site URL
        
        MyApps -> <your recently created app>
        
        Settings -> Website -> SiteURL
        
        for development:
        
        http://localhost:8000/
        
        this must match the url in browser when trying to do social login
        
        Note: http://127.0.0.1:8000/ will not work
      - Examples
        
        Facebook example in rauth
        
        How to make website registration easier using facebook accounts
    - Google
      - Google Identity Platform
    - Twitter
      - Twitter Developers
        
        Docs
        
        Basics
        
        Authentication
        
        OAuth with the Twitter APIs
        
        Application-only authentication: OAuth2 (bearer token)
        
        "Application-only authentication is a form of authentication where an application makes API requests on its own behalf, without the user context. This method is for developers that just need read-only to access public information."
        
        endpoints:
        
        POST oauth2/token
        
        POST oauth2/invalidate_token
        
        Application-user authentication: OAuth 1a (access token for user context)
        
        "The user authentication method of authentication allows an authorized app to act on behalf of the user, as the user."
        
        endpoints:
        
        POST oauth/request_token
        
        GET oauth/authorize
        
        POST oauth/access_token
        
        4.1. OAuth 1.0
        
        Media
        
        Upload Media
        
        Developer utilities
        
        Twitter libraries
        
        large-video-upload-python
      - Sign in with Twitter
      - Create an application
      - Problemes / Problems
        
        Preflight and CORS
        
        Will Twitter API support CORS Headers soon?
        
        Twitter API authorization fails CORS preflight in browser
        
        Solucions / Solutions
        
        How to turn nginx into a caching, authenticated Twitter API proxy (bearer token)
      - Angular JS
        
        OAuth2 with AngularJS to access Twitter
        
        OAuth2 with NodeJS , Express and AngularJS to access Twitter
  - ...

Mozilla
- Introduction to SSL
- Introduction to Public-key cryptography

Servidors de correu / E-mail servers

Infraestructura / Infrastructure
- Email agent (infrastructure) (wp)
  - Message Transfer Agent (MTA)
  - ...
- How email works (MTA, MDA, MUA)
- A short introduction to the notorious MxA bunch
- RFC 5598: Internet Mail Architecture
  - p23: Figure 5: Protocols and Services

DNS

Name	Type	Value
`example.org.`	`MX`	`1 mail.example.org`
`example.org.`	`TXT`	`"v=spf1 include:... include:..."`
`dddk._domainkey.example.org.`	`TXT`	`"v=DKIM1; k=rsa; p=..."`

Ports:

	non-secure (can be upgraded to secure by using STARTTLS)		secure (use STARTTLS instead)
output	SMTP	25, 587 (RFC 2476)	SMTPS	465 (revoked)
input	IMAP	143		993
input	POP3	110		995

Seguretat / Security
- Seguretat / Security
  - Info
    - SSL vs TLS vs STARTTLS
    - The Current State of SMTP STARTTLS Deployment
  - SSL / TLS (fixed ports: 465, 993, 995) -> use STARTTLS instead
  - STARTTLS (wp: Opportunistic TLS)
    - passa de no segur a segur / upgrade plain text to secure
    - no calen ports específics / no need for specific secure ports
    - es pot fer servir amb: / can be used with: SMTP, IMAP, POP3, ...
- DMARC
  - Overview
  - About DMARC (Google)
- SPF - Sender Policy Framework
  - Llistat de servidors que estan autoritzats a enviar missatges en nom del domini / List of servers authorized to send messages on behalf of the domain
  - SPF Record Syntax
  - Validator
  - Validate for GSuite
  - About SPF records (Google)
  - DNS TXT record:
    - "v=spf1 include:..."
- DKIM - DomainKeys Identified Mail
  - Signatura digital de les capçaleres / Digital signature of headers
    - clau privada: ...
    - clau pública: DNS TXT record
  - About DKIM (Google)
  - DKIMCore
    - Tools
      - Generate a DKIM Core Key
  - DNS TXT record:
    - "v=DKIM1; k=rsa; p=..."
- Verificació / Verification
  - Begin signing mail messages with the domain key
    - 6. To confirm that DKIM signing is active, send an email message to someone who is using Gmail or G Suite.
    - received message in Gmail:
      - "Show original"
        
        SPF: PASS
        
        DKIM: PASS
Implementacions / Implementations
- Postfix
- Dovecot

Clients de correu electrònic / E-mail clients

Obrir un adjunt winmail.dat / Open a winmail.dat attachment:
- Install tnef:
  - urpmi tnef
  - apt-get install tnef
- Unpack attachement content:
  - tnef winmail.dat
Command line
- How do I test an imap server?
- Testing POP3 and IMAP servers from the command line in CMD or bash
- imap:
  - telnet mail.example.com 143
  - ...
- telnet/nc to an SMTP server (after HELO, responses starting with 250 show available commands) (cannot continue a connection when STARTTLS has been invoked; use openssl s_client or mailx instead)
  - port 25 (default)
    - HELO (basic commands)
      - $ telnet mail.example.org 25 Trying x.x.x.x... Connected to mail.example.org. Escape character is '^]'. 220 client.example.org ESMTP Postfix HELO mail.example.org 250client.example.org MAIL FROM:<localuser@example.org> 250 2.1.0 Ok RCPT TO:<remoteuser@example.org> 250 2.1.5 Ok DATA 354 End data with <CR><LF>.<CR><LF> To:<remoteuser@example.org> From:<localuser@example.org> Subject:First test Hi, you! . 250 2.0.0 Ok: queued as 489C411EE7AC QUIT 221 2.0.0 Bye Connection closed by foreign host.
    - EHLO (enhanced commands)
      - $ telnet mail.example.org 25 Trying x.x.x.x... Connected to mail.example.org. Escape character is '^]'. 220 client.example.org ESMTP Postfix EHLO mail.example.org 250-client.example.org 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-STARTTLS 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN ...
  - port 587 (must be activated; see Postfix on port 587) and STARTTLS:
    - $ telnet mail.example.org 587Trying x.x.x.x... Connected to mail.example.org. Escape character is '^]'. 220 client.example.org ESMTP Postfix EHLO mail.example.org 250-client.example.org 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-STARTTLS 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN STARTTLS ...
- openssl s_client
  - openssl s_client -connect mail.example.org:587 -starttls smtp
  - openssl s_client -starttls smtp -connect smtp.gmail.com:587 -crlf -ign_eof
  - Problemes / Problems
    - read:errno=0 (and connection is closed):
      - check, on postfix server: /var/log/maillog
- mail / mailx
  - Instal·lació / Installation
    - CentOS
      - sudo yum install mailx
  - Utilització / Usage
    - Linux and Unix mailx command
    - mailx(1) - Linux man page
    - How can I use the “mail” command?
    - enviament / sending (a local email server, e.g. Postfix, must be running)
      - Sending Email Using mailx in Linux Through Internal SMTP
      - mail -s "subject" [-r from@toto.org] user@example.org
      - Note: some destination require a from address (-r) from a valid domain (MX entry). This is not the case for a destination address in gmail.com
    - enviament via un servidor smtp remot (p.ex. Postfix) / sending though a remote smtp server (p.ex. Postfix)
      - Sending Email from mailx Command in Linux Using Gmail’s SMTP
      - mail -s "subject" -S from=my_user@toto.org -S smtp=smtp://mail.toto.org destination_user@example.org
      - STARTTLS and authentication
        
        mailx -v -s "$EMAIL_SUBJECT" -S smtp-use-starttls -S ssl-verify=ignore -S smtp-auth=login -S smtp=smtp://smtp.gmail.com:587 -S from="$FROM_EMAIL_ADDRESS($FRIENDLY_NAME)" -S smtp-auth-user=$FROM_EMAIL_ADDRESS -S smtp-auth-password=$EMAIL_ACCOUNT_PASSWORD -S ssl-verify=ignore -S nss-config-dir=~/.mozilla/firefox/yyyyyyyy.default/ $TO_EMAIL_ADDRESS
    - recepció / reception
      - MailX Tutorial
      - mail
        
        next page: z
        
        previous page: z-
        
        delete: d<from>-<to>
        
        list of messages (headers): h
        
        ...
Mozilla Thunderbird (correu/mail)
- Command line options
  - create a new profile
    - thunderbird -CreateProfile my_profile
    - it will be created at:
      - ~/.thunderbird/xxxxxxxx.my_profile/
  - open gui profile manager
    - thunderbird -P
  - open a profile
    - thunderbird -p my_profile
- Thunderbird 3
- Redacció d'un missatge en HTML quan per omissió és en text pla / Compose a message in HTML when default is plain text:
  - Majúscula + click a "Redacta" / Shift + click on "Compose"
- Expanded columns in folder pane
- Signatures - Thunderbird
- enllaços cap a firefox i altres aplicacions: ~/.thunderbird/.../prefs.js:
  - Opening hyperlinks from some GTK-based applications doesn't work after updating Firefox to a new version
  - Bug 58784 - Using "Make firefox the default web browser" should use /usr/bin/firefox not /usr/lib(64)/firefox-<version>/firefox
  - $ gconf-editor
    - /desktop/gnome/url-handlers/http/command
      - enabled: no
      - com que el valor per omissió és "enabled=true", això crearà el fitxer .gconf/desktop/gnome/url-handlers/http/%gconf.xml (si no, no existeix el fitxer)
    - llavors el thunderbird ens preguntarà amb quina aplicació volem obrir els http: /usr/bin/firefox
  - gconf: ~/.gconf/desktop/gnome/url-handlers/http/%gconf.xml
  - Al Firefox: Edita / Preferències / Avançat / General / Comprova-ho ara (predeterminat) (*)
  - user_pref("network.protocol-handler.app.http", "/usr/bin/xdg-open");
  - user_pref("network.protocol-handler.app.http", "/usr/bin/mozilla-firefox");
  - user_pref("network.protocol-handler.app.https", "/usr/bin/mozilla-firefox");
  - user_pref("network.protocol-handler.app.ftp", "/usr/bin/mozilla-firefox");
  - user_pref("network.protocol-handler.app.smb", "/usr/bin/konqueror");
- Locale-Switcher Extension
- Tips & tricks
- Consells i trucs (Softcatalà)
- Use different Quote Level Colors
  - ~/.thunderbird/xxx.default/chrome/userContent.css
    - /* Quote Levels Colors */ blockquote[type=cite] { color: navy !important; background-color: RGB(245,245,245) !important; } blockquote[type=cite] blockquote { color: maroon !important; background-color: RGB(235,235,235) !important; } blockquote[type=cite] blockquote blockquote { color: green !important; background-color: RGB(225,225,225) !important; } blockquote[type=cite] blockquote blockquote blockquote { color: purple !important; background-color: RGB(215,215,215) !important; } blockquote[type=cite] blockquote blockquote blockquote blockquote { color: teal !important; background-color: RGB(205,205,205) !important; }
- Problem: "This body part will be downloaded on demand"
  - View->Display Attachments Inline
- how to not include external images on html messages? - Mozilla
  - moz-do-not-send true
  - En recepció, caldrà acceptar "Mostra el contingut remot"
- Lectura de fitxers mbox / Read mbox files
  - Reading an mbox file with Thunderbird
  - min_thunderbird.sh
    - #!/bin/bash EXPECTED_ARGS=1 if (( $# != $EXPECTED_ARGS )) then cat <<EOF Usage: `basename $0` profile_name Create a thunderbird profile with given name, and add a default mail setup, in order to read mbox files. EOF exit 1 fi # check that thunderbird is installed if ! command -v thunderbird2 >/dev/null 2>&1 then echo "ERROR: command thunderbird does not exist. Please install it (https://www.thunderbird.net/) before running this script." exit 1 fi profile_name=$1 root_thunderbird_dir=${HOME}/.thunderbird # create profile echo "1. creating profile: ${profile_name}" thunderbird -CreateProfile ${profile_name} # get profile path profile_dirname=$(find ${HOME}/.thunderbird -type d -name "*\.${profile_name}") echo " created: ${profile_dirname}" # start thunderbird, to create dirs echo "2. starting thunderbird to create dirs: please just cancel and close" thunderbird -p ${profile_name} # backup orginal prefs.js prefs_path=${profile_dirname}/prefs.js original_prefs_path=${profile_dirname}/prefs.original.js echo "3. backing up ${prefs_path} -> ${original_prefs_path}" cp ${prefs_path} ${original_prefs_path} # modify prefs.js echo "4. modifying ${prefs_path} to add a minimal mail config" cat >>${prefs_path} <<EOF user_pref("mail.root.none", "${profile_dirname}/Mail"); user_pref("mail.accountmanager.accounts", "account1"); user_pref("mail.accountmanager.localfoldersserver", "server1"); user_pref("mail.account.account1.server", "server1"); user_pref("mail.server.server1.directory", "${profile_dirname}/Mail/Local Folders"); user_pref("mail.server.server1.directory-rel", "[ProfD]Mail/Local Folders"); user_pref("mail.server.server1.hostname", "Local Folders"); user_pref("mail.server.server1.name", "Carpetes locals"); user_pref("mail.server.server1.nextFilterTime", 10); user_pref("mail.server.server1.spamActionTargetAccount", "mailbox://nobody@Local%20Folders"); user_pref("mail.server.server1.storeContractID", "@mozilla.org/msgstore/berkeleystore;1"); user_pref("mail.server.server1.type", "none"); user_pref("mail.server.server1.userName", "nobody"); EOF # start thunderbird echo "5. starting thunderbird: just check that default mail folders have been created and close it" thunderbird -p ${profile_name} # additional mbox files echo "6. to read mbox files, put them on (or link them from) ${profile_dirname}/Mail/Local Folders/ and restart thunderbird by using one of the following:" echo " thunderbird -p ${profile_name}" echo " thunderbird -P" echo "To delete profile ${profile_name}:" echo " thunderbird -P" echo "and select Delete Profile..." echo "IMPORTANT: if you made a symbolic link to your mbox file from ${profile_dirname}/Mail/Local Folders/, you can choose Delete Files option. Otherwise, choose Don't Delete Files." exit 0
Gmail
- Priority box
- API
  - Reference
  - Python
    - Python quickstart
    - ...

Navegadors / Browsers

List of web browsers (wp)
Browserscope
Browsers.com
Herramientas para una navegación satisfactoria (La Vanguardia)
"A virtually secure browser" (pdf)
Web browser standards support
Automatització / Automation
- Selenium
User agents
- What is my user agent?
- List of User Agent Strings
- Changing User Agent in Firefox
  - general.useragent.override
Plug-ins
- Plugger (Streaming Multimedia Plugin for Unix Netscape)
- Netscape Inline Plug-ins
- ViewDirector PRIZM Sample Image Page
- Flash
- PNG Features
- RealAudio
- Stream Works (test)
- UNIX MIDI Plugin for Netscape
- Xing Technology Corporation
- CosmoPlayer
- mplayer

Configuració / Settings

Reference for all “about”/“chrome” address bar strings for Firefox and Chrome

		chrome://	about:
		Chrome	Firefox
Llista de les urls internes disponibles		`chrome://chrome-urls/`
		`chrome://flags`	`about:config`
Preferències / Settings		`chrome://settings`	`about:preferences`
	Adobe Flash	`chrome://settings/content/flash`	`about:addons`
	Galetes / Cookies	`chrome://settings/content/cookies`	`about:preferences#privacy`
Media		`chrome://media-internals/`
Components/Complements / Add-ons		`chrome://components/`	`about:addons`
WebRTC		`chrome://webrtc-internals/`	`about:webrtc`
		`chrome://whats-new/`

Mozilla (*)
- Mozilla Firefox (navegador/browser)
  - Trucs / Tips
    - reinicia (manté pestanyes i finestres) / restart
      - about:profiles
        
        Reinicia normalment...
    - rodeta del ratolí per a canviar de pestanya / use wheel mouse to switch tabs:
      - about:config
        
        toolkit.tabbox.switchByScrolling: true
      - reinicia / restart
    - decoració de la finestra amb títol (KDE)
      - menú hamburguesa (dalt dreta) -> Més eines -> Personalitza la barra d'eines
        
        (a baix a l'esquerra) Barra de títol
    - ...
  - Firefox nightly builds
  - Working with Mozilla source code
  - Build instructions
  - Debug
    - real Android device
      - Debugging Firefox for Android over USB (Firefox >=36)
      - Debugging Firefox for Android over WiFi (Firefox >=42)
  - Old releases download (ftp)
    - Differences between Win32 and Win32-EUballot builds
  - Firefox Beta
    - 4.06b Release notes
  - Mozilla Firefox 3.6
    - Firefox 3.6 for developers
      - HTML5
        
        video
  - Mozilla Firefox 3.5 (developer)
    - Firefox 3.5 Beta 4 Release Notes
    - Five Game-Changing Features in Firefox 3.5
    - HTML5
    - Audio/video tags
      - tinyvid.tv
    - Mozilla previews new feature to guard against Flash crashes
  - Electrolysis
  - Mozilla Developer Preview
  - Keyboard shortcuts
  - How To Manage Profiles
  - 1.0 linux xpi
  - HTML VALIDATOR (based on Tidy)
  - Extensions
  - - Web developer
    - Extension room
    - - Digger (site navigation bar)
    - Comment choisir son extension Firefox
    - Link toolbar (*)
    - GrouperParCritère
    - Ubiquity
    - KidZui (navigation for children)
    - Geolocalització / Geolocation
    - DOM Inspector
    - Konquefox: clear URL button
    - Cooliris
    - OptimizeGoogle
    - CustomizeGoogle (*)
    - Video Download Helper
    - Fire HbbTV
    - FireBug
    - Poster (GE, POST, ...)
    - Markdown
      - markdown-viewer
        
        Installing on Linux
        
        ~/.local/share/mime/packages/text-markdown.xml
        
        update-mime-database ~/.local/share/mime
      - GitLab Markdown viewer
  - Kiosk
    - Speed Dial (*)
  - Print
    - Personalització / Customization
      - about:config
        
        print.print_headerleft: &T (title)
        
        print.print_headercenter:
        
        print.print_headerright: &U (url)
        
        print.print_footerleft: &PT (page counter)
        
        print.print_footercenter:
        
        print.print_footerright: &D (date)
    - KDE Printing in Mozilla
  - Closing tabs
    - about:config
      - browser.tabs.closeButtons 3
      - browser.tabs.insertRelatedAfterCurrent true
  - Open file:///// from a web page:
    - about:config
      - security.fileuri.strict_origin_policy false
    - LocalLink plugin
    - Links to local pages do not work
  - Double/single click in url
    - about:config
      - browser.urlbar.clickSelectsAll false
      - browser.urlbar.doubleClickSelectsAll true
- Mozilla Thunderbird
- Mozilla Evangelism
- mozilla.org Bugzilla
- MozDev
- mozillaZine
- MozillaNews
- Bug 74157: signing and encrypting messages
- Mozilla's DOCTYPE sniffing (strict vs quirk mode)
- User agent
  - uabar
- Mozilla Font Size / DPI Issues on Unix
- - chrome/userChrome.css (UI)
- Customizing Mozilla
- Ongoing Localization Projects
  - Mozilla Softcatalà
  - català: xpi-1.7b-rc2
- signForm implementation
- Cannot do formsigning - crypto.signText (bugzilla)
- secclab
- moz-do-not-send = true
Opera
- Opera TV Emulator (download)
  - ln -s /usr/lib/libpng12.so /usr/lib/libpng14.so.14
WaMCom (signText)
El Navegador
- El Navegador (Softcatalà)
- Enciclopèdia Catalana (obsolet)
Netscape
- at Switch
- SGI Netscape
- Remote Control of UNIX Netscape
- The Netscape Unofficial FAQ
  - Additional languages to the spell checker
- My Netscape FAQ
- Internet Technologies
- DevEdge Online
  - Documentation & Information
    - Manuals
- Communicator Release Notes
- Crash on Linux
- Old gnumalloc.so to workaround Netscape crash on linux (Troubleshooting plugger)
- Netscape Developer Release Notes (v6.x)
NCSA Mosaic
Konqueror
Google
- Chromium
  - Berkelium
- Chrome
  - Trucs
    - decoració de la finestra amb títol (KDE)
      - menú tres pics verticals (dalt a la dreta) -> Configuració
        
        Aparença -> Utilitza la barra del títol i els límits del sistema
  - How To Install Google Chrome 41 (41.0.2272.118) On The Most Popular Linux Systems, Via The Official Google Repository
  - Mageia
    - rpm --import https://dl-ssl.google.com/linux/linux_signing_key.pub
    - urpmi.addmedia --update chrome_x86_64 http://dl.google.com/linux/chrome/rpm/stable/x86_64
    - urpmi google-chrome-stable
  - Internal addresses
    - chrome://components/
    - chrome://media-internals/
    - chrome://settings/content/flash
  - Adobe Flash
  - Single/Double click in url bar
    - Disable select all text on single click in address bar
    - Linux Aura: Left-click in address bar should not select all text
  - Settings
    - Galetes / Cookies
  - Extensions
    - HLS
      - Native HLS Playback
        
        based on hlsjs
        
        from file:
        
        ...
  - Debug
    - Media (e.g. hls.js)
      - chrome://media-internals/
Microsoft Internet Explorer
- Crash
- Stop Using Internet Explorer!
- Internet Explorer Vulnerability Report (Secunia)

Command line

openssl

curl

curl apis
- PHP
  - curl_setopt
- Python

opcions / options:

group	option	description
input	`-X http_method`	method
input	`-H http_request_header`	request header (e.g. to specify origin for CORS)
output	`-i`	include the HTTP response header in the output
	`-I`	displays only the HTTP response header in the output
	`-s, --silent`	silent
	`-v`	verbose (show also HTTP request headers)
	`-O`	write output to a local file named like the remote file we get (like wget)
	`-o, --output <local_file>`	write output to a local file named as specified
	`-L`	follow new location (3xx)
	`-m, --max-time <seconds>`	Maximum time in seconds that you allow the whole operation to take
	`-w, --write-out <format>`	Make curl display information on stdout after a completed transfer.
	`--referer` ...	referer
authentication	`-u user:password`
galetes / cookies	`-b filename / <name>=<value>`	read from file / send a single cookie
	`-j`
	`-c` `filename`	write to file
https	`-k`	trust certificate
https	`--cacert ca_certificate`	specify a certificte for a trusted CA
	...

exemples / examples:
- curl -X GET -H 'Accept: text/plain' http://example.com/
- curl -X GET -H 'Accept: application/json' http://example.com/
- curl -X GET -u usuari:contrasenya http://example.com/
- POST data
  - curl -X POST --data 'toto=true' http://example.com/my_api/
  - JSON
    - curl -X POST -H 'Content-Type: application/json' --data-binary '{"toto":true}' http://example.com/my_api/
    - curl -X POST -H 'Content-Type: application/json' --data-binary '@toto.json' http://example.com/my_api/
    - my_value="toto" my_json='{ "my_key":"'${my_value}'", "my_other_key": "my_other_value" }' echo ${my_json} | curl ... -H 'Content-Type: application/json' -d @-
    - jq -cM toto.json | curl ... -H 'Content-Type: application/json' -d @-
    - nested json as multipart (e.g.: to be able to upload an image at the same time)
      - curl -X POST -H 'Content-Type: multipart/form-data' -F name='nom' -F location.name='Reus' -F location.point='{"type": "Point","coordinates": [2.084205,41.473491]}}' -F thumbnail=@cover.pnghttp://example.com/my_api/
- GET JSON
  - curl -H 'Accept: application/json; indent=4' -u admin:password http://127.0.0.1:8000/users/
  - curl -H 'Accept: application/json' -u admin:password http://127.0.0.1:8000/users/ |python -m json.tool
  - curl -X OPTIONS -H 'Accept: application/json; indent=4' -u admin:password http://127.0.0.1:8000/users/
- curl -I -X GET http://example.com/
- HEAD
  - curl --head ...
- get modification time:
  - curl --head http://example.org/toto 2>/dev/null | awk '/Last-Modified/ {$1="";print}'
- condition based on modification time:
  - curl --time-cond ...
  - ...
- descarega i desa un fitxer / download and save a file
  - amb el mateix nom / with the same name:
    - curl -O https://.../toto.txt
  - amb un nom diferent / with another name:
    - curl -o tata.txt https://.../toto.txt
- upload a file
  - curl --upload-file my_file.png http://...
  - curl -F filedata=@localfile.jpeg http://example.org/upload
  - specifying the filename that will be used to store the file in server:
    - curl -F filedata="@localfile.jpeg;filename=remotefile.jpeg" http://example.org/upload
    - Specify file name with curl --upload-file
- percent encoding
  - How to urlencode data for curl command?
  - post params
    - # equivalent to https://... with post params curl ... --data-urlencode "myparam=myvalue" ...
  - query params
    - # equivalent to https://.../?myparam=myvalue_urlencoded curl ... --get --data-urlencode "myparam=myvalue" ...
- Authentication
  - Django authentication
  - # login user1 key=$(curl -X POST -H 'Content-Type: application/json' --data-binary '{"username":"user1","password":"mypassword"}' http://example.org/rest-auth/login/ | awk -F ':' '/key/{gsub(/[}"]/,"",$2); print $2}') # get user details curl -X GET -H 'Accept: application/json; indent=4' -H "Authorization: Token $key"http://example.org/rest-auth/user/
- CORS
  - In order to get a response header , you must specify an Origin:
    - curl -i -H "Origin: http://example.com" ...
      - to make sure that this url will work from a browser, you should get the following response headers:
        
        Access-Control-Allow-Origin
        
        ...
  - How can you debug a CORS request with cURL?
    - regular CORS
      - curl -H "Origin: http://example.com" --verbose ...
    - preflight CORS
      - curl -H "Origin: http://example.com" \ -H "Access-Control-Request-Method: POST" \ -H "Access-Control-Request-Headers: X-Requested-With" \ -X OPTIONS --verbose ...
- referer
  - curl --referer "http://example.org/index.html" ...
- download a file honouring Content-Disposition
  - How to use content-dispositon header to determine filename
  - curl --remote-name ...
- Galetes / Cookies
  - HTTP Cookies
  - start cookie engine and read/write cookies to a file
    - write to file
      - -c galetes.txt
    - read from file
      - -b galetes.txt
  - send a single cookie
    - -b MY_COOKIE=ABCD12345
- If-Modified-Since
  - check if the origin has a resource newer than a specified date
    - curl -I -z "Fri, 16 Sep 2016 09:43:09 +0200" ...
    - curl -I -z "$(date -R -d 2016-09-16T07:43:09,328834376+0000)" ...
    - curl -I -H "If-Modified-Since $(date-R -d 2016-09-16T07:43:09)" ...
- write-out
  - print times and latencies
    - How do I debug latency issues using curl?
    - curl -w "dns_resolution: %{time_namelookup}, tcp_established: %{time_connect}, ssl_handshake_done: %{time_appconnect}, TTFB: %{time_starttransfer}\n" ...
- reintents / retries
  - # retry: # - every attempt will wait for up to 10 seconds (--max-time 10) # - wait for 5 seconds before trying a new attempt (--retry-delay 5) # - up to 3 times (--retry 3) # maximum total time: 10 + (first retry:)5+10 + (second retry:)5+10 + (third retry:)5+10 = 55s retry_parms = "--connect-timeout 5 --max-time 10 --retry 3 --retry-delay 5 --silent --show-error"
- get http response code from bash
  - How to split the HTTP error code from the contents in cURL?
  - How to evaluate http response codes from bash/shell script?
  - --write-out
    - Using curl’s –write-out option to print extra information about a request
  - Example:
    - # add http_code as an extra final line, after the response OUT=$(curl --silent --write-out '\n%{http_code}' -X GET http://www.example.org/) # get the output code of the curl command RET=$? if [[ $RET -ne 0 ]] then     echo "Error $RET" else     # response is everything but the last line     response=$(echo "$OUT" | head -n-1)     echo "Response: $response"     # http code is the last line        http_code=$(echo "$OUT" | tail -n1)     echo "http code: $http_code"
          if [[ $http_code == "200" ]]     then         # ok         ...     else         # not ok         echo "response: $response"         exit$http_code
          fi fi
  - several retries (bash function)
    - #!/bin/bash function send_curl { # parameters: # method "url" "options" # output is kept in global variable curl_response # return value: # 0: ok (2xx) # 1: reached maximum number of retries # other https codes return the sum of digits. e.g.: # 4: 400 # 7: 403 # 8: 404 # ... # usage examples: # send_curl GET "http://192.168.1.100:8000/path/to/?toto=1&tata=2" "-H 'Authorization: Token $key'" # send_curl POST "http://192.168.1.100:8000/path/to/" "-H 'Authorization: Token $key' -F tete='titi'" local method=$1 local url=$2 local options=$3 echo "method: $method" echo "url: $url" echo "options: $options" local curl_command="curl --silent --write-out '\n%{http_code}' -X $method $options '$url'" echo "curl_command: $curl_command" local return_code=1 local max_retries=10 local retries=0 curl_response="" while (( retries < max_retries )) do echo "[$retries] curl_commmand: ${curl_command}" OUT=$(eval "$curl_command") RET=$? if [[ $RET -ne 0 ]] then echo "Error $RET" else http_code=$(echo "$OUT" | tail -n1) echo "http code: $http_code" curl_response=$(echo "$OUT" | head -n-1) echo "curl_response: $curl_response" # 2xx if [[ $http_code =~ ^"2" ]] then echo "ok" return_code=0 else # return code is the sum of digits of http_code return_code=$(expr $(echo $http_code | sed 's/[0-9]/ + &/g' | sed 's/^ +//g')) echo "not ok: http_code=$http_code -> return_code=$return_code" fi break fi sleep 2 (( retries++ )) done return $return_code }
  - json paginated result (e.g. from Django rest framework result)
    - #!/bin/bash method=$1 url=$2 options=$3 function get_page { local method=$1 local url=$2 local options=$3 # execute curl request curl_command="curl --silent --write-out '\n%{http_code}' -X $method $options '$url'" echo "${curl_command}" OUT=$(eval "$curl_command") RET=$? # http response code http_code=$(echo "$OUT" | tail -n1) # http response curl_response=$(echo "$OUT" | head -n-1) # total number of elements, including all pages curl_count=$(echo ${curl_response} | jq '.count') # link to next page curl_next=$(echo ${curl_response} | jq -r 'if .links.next then .links.next else empty end') } function get_all_pages { local method=$1 local url=$2 local options=$3 # first request get_page "${method}" "${url}" "${options}" retrieved_elements=$(echo ${curl_response} | jq '.results') cumulated_json="${retrieved_elements}" # subsequent requests, until no next is found while [[ "${curl_next}" ]] do get_page "${method}" "${curl_next}" "${options}" retrieved_elements=$(echo ${curl_response} | jq '.results') # add new results to cumulated results cumulated_json=$(echo ${cumulated_json} | jq ". += ${retrieved_elements}") done echo "cumulated json:" echo "${cumulated_json}" | jq '' } get_all_pages "${method}" "${url}" "${options}" exit 0
  - mime-type / content_type
    - curl_get_command="curl --silent --write-out '\n%{content_type}\n%{http_code}' -X GET ..." OUT=$(eval ${curl_get_command}) RET=$? echo "OUT: $OUT" if [[ $RET -ne 0 ]] then echo " Error $RET" exit $RET else content_type=$(echo "$OUT" | tail -n2 | head -1) echo " content_type: $content_type" http_code=$(echo "$OUT" | tail -n1) echo " http_code: $http_code" response=$(echo "$OUT" | head -n-1) if [[ $http_code == "200" ]] then echo " ok" else echo " response: $response" fi fi
- AWS user data
  - # get user-data # check if there is user_data http_code=$(curl --write-out '%{http_code}\n' --head -o /dev/null -s http://169.254.169.254/latest/user-data/) if [[ $http_code == "200" ]] then user_data=$(curl -s http://169.254.169.254/latest/user-data/) ... else ... fi
- Django REST framework from curl
- Django REST framework test
- Django Tastypie from curl
HTTPS
- cURL: Adding/Installing/Trusting New Self-Signed Certificate
- per a connectar-vos amb un servidor amb un certificat de servidor emès per una CA / to connect to a server with a server certificate issued by a CA:
  - curl ... --cacert ca_that_issued_the_server_certificate.crt ... https://...
- per a connectar-vos amb un servidor amb un certificat autosignat / to connect to a server with a selfsigned server certificate
  - aconseguiu el certificat de servidor / get the server certificate (autosigned_server_certificate.crt)
  - curl ... --cacert autosigned_server_certificate.crt ... https://...
- o bé afegiu el certificat de la CA (o el certificat autosignat del servidor) a la llista de certificats de confiança / or add the CA certificate (or the selfsigned server certificate) to the list of trusted certificates:
  - sudo cp myserver.example.org.crt /etc/pki/ca-trust/source/anchors/
  - sudo update-ca-trust
  - cURL: Adding/Installing/Trusting New Self-Signed Certificate
  - Certificat de servidor amb Fabric / Server certificate with Fabric
- trust (selfsigned) certificates:
  - curl -k ...
- Issues with Let's Encrypt certificates
mesura del temps / time measure
- How do I measure request and response times at once using cURL?
- curl-format.txt
  - time_namelookup: %{time_namelookup}s\n time_connect: %{time_connect}s\n time_appconnect: %{time_appconnect}s\n time_pretransfer: %{time_pretransfer}s\n time_redirect: %{time_redirect}s\n time_starttransfer: %{time_starttransfer}s\n ----------\n time_total: %{time_total}s\n
- curl -w "@curl-format.txt" -o /dev/null -s "https://.../"

GNU Wget (at GNU)
- galetes / cookies
  - Cookie Exporter (Firefox add-on)
    - cookies.txt
  - wget --load-cookies=cookies.txt ...
  - Export cookies Firefox add-on
  - Using the cookies.sqlite from Firefox 3 in wget
    - urpmi sqlite3-tools
  - How to recover a corrupted Firefox cookies.sqlite
- HTTPS
  - client authentication:
    - wget --certificate=client_pem.crt --private-key=client_pem.key --ca-certificate=root_ca_pem.crt https://...
- recursiu / recursive:
- - wget -c -np -r --level=5 -e robots=off http://...
  - How to download recursively from an FTP site
Clients FTP
WGET software for FTP and Web Auto-mirroring (alternatives)
Snarf
Pavuk
W3M

Java-based
- X-Smiles (XML-browser in Java)
  - Seguretat
    - fi.hut.tml.xsmiles.util.HTTPClient.ssl
  - OpenTDT (MHP)

HTML

Accessibilitat / Accessibility
Characters, entities
- ISO 8859-1 (Latin-1) Characters List
- Entity References Test Document
- USPTO Patent Application V1.5 2001-01-31: User Entity View (llatí: macron, breve...)
- Dave Raggett's Introduction to HTML: entities
- ttfenc.c
- convert to html entity:
  - echo "aé" | recode ..html
Desenvolupament / Development
- General structure of a site
  - HTML5 Boilerplate (basic stucture of a site)
    - Includes
      - Normalize.css
      - jQuery
      - Modernizr (features detection)
      - Server Configs
        
        Apache
        
        nginx
        
        ...
      - EditorConfig (coding styles)
    - Custom builds
      - Initializr (HTML5 templates generator)
        
        Initializr - great HTML5 Boilerplate startup template
        
        Pre-configurations
        
        Classic HTML5 Boilerplate
        
        Responsive
        
        Bootstrap
- HTML/CSS Templates
  - Bootstrap (2.3.2) (HTML/CSS Template) (by Twitter)
    - Bootstrap studio

Editors

	edit html code	wysiwyg	active	last update
Amaya (w3c)				2012
Aptana Studio 3 Core			not active, but open source	3.7.2 (2018-07-30)
Atom			discontinued (2022-12-15)
Blogilo (KDE)			discontinued
Bluefish (gnome)	x
Blue Griffon	x	x	discontinued
Eclipse
Emacs	x		x
Google Web Designer	only ads
kompozer			discontinued (replaced by bluegriffon)	2010-09-26
Maqetta
nvu			discontinued (replaced by kompozer)	2005-06
Quanta (kdewebdev)			discontinued	2007-11-09

Llistes
- Eclipse plugin for HTML WYSIWYG editor [closed]
  - JavaServer Faces(JSF) Tools Project
  - JBoss Tools
    - HTML
  - MyEclipse (commercial)
  - LeeEdit
- Mag's Big List of HTML Editors
AdvaSoft AsWedit
- MDK compatibility
ASHE
tkHTML (tcl/tk)
HTML helper mode for Emacs
Quanta
Blue Griffon
- Installation
  - Mageia 9
    - install tgz version, built for ubuntu
  - Mageia 3
    - bluegriffon-1.6.2-3.mga3.mrc.x86_64.rpm
- Add-ons
  - Download
  - Install
    - Tools / Add-ons / engranatge/cogwheel (Install Add-on from file)
      - fireftp-2.0.32-bluegriffon.xpi
- Source
  - therealglazou/bluegriffon
- Compilation
  - Build BlueGriffon
  - dependencies
    - Mageia 9
      - sudo dnf install rust cargo autoconf2.1 lib64gtk+2.0-devel lib64GConf2-devel
    - Mageia
      - urpmi autoconf2.1
  - Passos
    1. cd ~/src
    2. get gecko-dev (~9,4 GiB)
      - git clone https://github.com/mozilla/gecko-dev bluegriffon-source
    3. get bluegriffon
      - cd bluegriffon-source
      - git clone https://github.com/therealglazou/bluegriffon
    4. update mozilla tree
      - git reset --hard `cat bluegriffon/config/gecko_dev_revision.txt`
      - patch -p 1 < bluegriffon/config/gecko_dev_content.patch
      - patch -p 1 < bluegriffon/config/gecko_dev_idl.patch
    5. create .mozconfig (e.g. for ubuntu or other linux)
      - ln -s bluegriffon/config/mozconfig.ubuntu64 .mozconfig
    6. dependencies
      1. Python 2.7
        
        pyenv install 2.7
        
        pyenv local 2.7
    7. build:
      - ./mach build
      - not working...
    8. ...
- Problemes / Problems
  - Bug 637 - Corrupted menus
    - Solució / Solution
      - No feu servir oxygen-gtk / Don't use oxygen-gtk (gtk-config)
  - Bug 636 - BlueGriffon is not useable on Debian 7
  - Bluegriffon Crash when clicking on any pull-down menu heading
Kompozer
- Kompozer labs
- HTML Timing
- Compilation
  - Kompozer Dev-Howto
  - dependencies
    - urpmi gcc-c++ lib64gnome-vfs2-devel lib64IDL2-devel lib64xt-devel lib64ftgl-devel lib64freetype2-devel lib64pangox-devel lib64png12-devel
  - svn checkout https://kompozer.svn.sourceforge.net/svnroot/kompozer/trunk kompozer
  - svn patch this.patch
  - kompozer/obj-kompozer/config/autoconf.mk
    - XT_LIBS = -lX11 -lXt
    - FT2_LIBS = -lfreetype -lftgl
  - cd kompozer
  - cp mozilla/composer/config/mozconfig.fedora mozilla/.mozconfig
  - make -f client.mk build_all
- Problemes / Problems (vegeu / see: Bluegriffon)
- Bugs
  - <object> makes title disappear from tab
Nvu
- replaced by kompozer
- Launchy
- Nvu Site Manager ConText
Atom
gwrite
- Compilation
  - dependencies
    - urpmi python-distutils-extra intltool
  - python-jswebkit
    - urpmi python-cython lib64webkitgtk1.0-devel lib64python-devel
    - wget https://gwrite.googlecode.com/files/python-jswebkit-0.0.3.tar.gz
    - tar xvzf python-jswebkit-0.0.3.tar.gz
    - cd python-jswebkit-0.0.3
    - su; python setup.py install
  - gwrite
    - wget https://gwrite.googlecode.com/files/gwrite-0.5.1.tar.gz
    - tar xvzf gwrite-0.5.1.tar.gz
    - cd gwrite-0.5.1
    - su; python setup.py install; ./install
Wix (Flash)
Aptana Studio 3 (Eclipse)
- prerequisites:
  - urpmi eclipse git
- Eclipse: Help -> Install new software -> Add
  - http://download.aptana.com/studio3/plugin/install
- Templates:
  - HTML5 Boilerplate
- Problems:
  - Previously installed PyDev must be uninstalled (*)
Javascript
- Webstorm (JetBrains)

Manuals and specifications

HTML 5
Metadata
- Info
  - Is there a standardized (meta?) tag for the date of a website?
HTML 4.01 Specification (w3c)
- LINK MEDIA: screen, tv, projection, handheld...
- Media types
- Media descriptors
- CSS2: Media types
HTML 4.0 Working Draft Release (w3c)
HTML 3.2 Features At a Glance (w3c)
A Beginner's Guide to HTML (NCSA)
Frames -An Introduction (Netscape)
Hagan's World of HTML (3.0)
HTML Documentation (University of Toronto)
Build Your Business Website

HTML META, REL and REV tags

HTML <meta> Tag
Redireccionament / Redirection
- Meta refresh
  - <meta http-equiv="refresh" content="5; url=http://example.com/">

attribute	value	info	example
`charset`	`character-set`	Character encoding of HTML document (HTML5)
`content`	`text`	Used by http-equiv, name
`http-equiv`	`content-type`	HTTP headers	`<meta http-equiv="content-type" content="application/vnd.hbbtv.xhtml+xml;charset=UTF-8">` `<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-15">`
	`default-style`
			`<meta http-equiv="Content-Style-Type" content="text/css">`
	`refresh`		`<meta http-equiv="refresh" content="5; url=http://example.com/">`
`name`	`application-name`	Metadata
	`author`
	`description`
	`generator`		`<meta name="generator" content="HTML Tidy for Linux/x86 (vers 1st December 2004), see www.w3.org">`
	`keywords`
	`viewport`		`<meta name="viewport" content="width=device-width">`
			`<meta name="qrichtext" content="1">`
	`robots`	The ultimate guide to the meta robots tag
`scheme`	`format/URI`	Used by content (HTML4)

Link
- The <link> Element in Mozilla's Site Navigation Bar
- Links types (w3.org: Basic HTML data types)
- Buttons in browser
  - Firefox
    - Link Widgets

MathML (wp)

MathML (MDN - Mozilla)
W3C Math Home
Brief Tutorial on MathML

Specification

	tokens	entities
Presentation MathML `<math mode="display" xmlns="http://www.w3.org/1998/Math/MathML"> </math>`	`<mi>x</mi>` – identifiers `<mo>+</mo>` – operators `<mn>2</mn>` – numbers `<mtext>non zero</mtext>` – text `<mrow>` – a horizontal row of items; `<msup>, <munderover>`, and others – superscripts, limits over and under operators like sums, etc.; `<mfrac>` – fractions; `<msqrt>` and`<mroot>` – roots; `<mfenced>` - surrounding content with fences, such as parentheses.	`π` `&RightArrow;` `⁢` `...`
Content MathML

Lletres / Fonts
- Fonts for Mozilla's MathML engine
- MathML-fonts (Mozilla Firefox add-on)

Editors

Conversion from a Simple Syntax


Client-side conversion	TeXZilla (live demo) (<x-tex>) (based on LaTeX) (local) base-commands.txt	<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <title>Example of x-tex</title> <link rel="stylesheet" type="text/css" href="css/x-tag-components.css"/> <link rel="stylesheet" type="text/css" href="css/tex.css"/> <script type="text/javascript" src="js/x-tag-components.js"></script> <script type="text/javascript" src="js/tex.js"></script> </head> <body> <h1>Example of x-tex</h1> <p>Inline example: <x-tex>\frac{x^2}{a^2} + \frac{y^2}{b^2} = 1</x-tex> </p> <p>Block example: <x-tex display="block">\frac{x^2}{a^2} + \frac{y^2}{b^2} = 1</x-tex> </p> </body> </html>	Inline example: \frac{x^2}{a^2} + \frac{y^2}{b^2} = 1 Block example:\frac{x^2}{a^2} + \frac{y^2}{b^2} = 1
	AsciiMath
	LaTeXMathML
Server-side conversion

Firemath - The Equation Editor (Mozilla Firefox add-on)

Navegadors / Browsers

TeXZilla en local
- Per a poder tenir tex.{css,js}, x-tag-components.{css,js} en local:
  - cd ~/src/
  - git clone https://github.com/fred-wang/x-tex.git
  - cd /path/to/html/root/css/
    - ln -s ~/src/x-tex/demo/x-tag-components.css .
    - ln -s ~/src/x-tex/src/tex.css .
  - cd /path/to/html/root/js/
    - ln -s ~/src/x-tex/demo/x-tag-components.js .
    - ln -s ~/src/x-tex/src/tex.js

Perquè tots els navegadors puguin presentar el codi escrit en MathML, cal afegir un pollyfill:

project	script	usage
	`<script src="https://fred-wang.github.io/mathml-warning.js/mpadded-min.js"></script>`	deixa triar a l'usuari
mathml.css	`<script src="https://fred-wang.github.io/mathml.css/mspace.js"></script>`
MathJax	`<script src="https://fred-wang.github.io/mathjax.js/mpadded-min.js"></script>`

Firefox
- Mathzilla: Collection of MathML-related add-ons

HTML5 (draft)(WhatWG Web applications 1.0) (wp)

HTML5 roadmap

Semantic

HTML4	HTML5
<b>	<strong>
<i>	<em>
<br/>	<br/>
<hr/>	<hr/>
	<ins>
line-through	<del>

HTML5 video
HTML5 audio
- Web audio examples (Chromium svn)
WebSocket
- HTML5 Real Time and WebSocket Code Lab (Slideshare)
- Kaazing WebSocket Gateway
Devices
- getUserMedia
  - More fun using the Web, with getUserMedia and native pages!
  - getUserMedia(), the device API and the state of the webcam in browser using javascript and html5 video
Navegadors / Browsers
- The HTML5 test (how well does your browser support HTML5?)
  - TV browsers
- Chromium: Web platform status
Tutorials
- Internet is hard - But it doesn't have to be (Atom editor)
  - Introduction
  - Basic web pages
  - Links and images
  - Hello, CSS
  - The box model
  - CSS selectors
  - Floats
  - Flexbox
  - Advanced positioning
  - Responsive design
  - Responsive images
  - Semantic HTML
  - Forms
  - Web typography
Differences from HTML4
HTML for Beginners - Ultimate Guide
Android Gets a Better Browser, Now With More HTML5
Introducing YouTube HTML5 Supported Videos
Can Flash Survive HTML5?
The Gradual Disappearance Of Flash Websites
canvas element (wp)
- canvas on Firefox
- canvas demos
3D
- 3D landscape in HTML5 canvas
- WebGL
Demos
- The wilderness downtown
- Sketchpad
- Slides (html5rocks)
Conversió / Conversion
- Swiffy, Google presenta su herramienta para convertir de Flash a HTML5

Estil / Style

CSS
Responsiu / Responsive
- 10. Responsive design (HTML & CSS is hard)
- Responsive web design
Esquemes de colors
- Color Scheme (w3schools)
Mode fosc / Dark mode
- segons les preferències d'usuari
  - prefers-color-scheme
    - cal posar, al final del document css:
      - @media (prefers-color-scheme: dark) { body { background: black none repeat scroll 0% 50%; color: white; } th.seccio-cap { background-color: #3399ff; } td.seccio-fons { background-color: #000000; } }
  - Emulació des del navegador
    - Firefox:
      - ...
    - Chrome
      - Més eines -> Eines per a desenvolupadors -> (tres punts verticals) -> Més eines -> Renderització -> Emula la funció multimèdia CSS prefers-color-scheme

HTML & CSS is hard

Introduction
Basic web pages
Links and images
- IANA Language subtag registry
- Character Entity Reference Chart
Hello, CSS
- MDN CSS Reference

CSS Box model

Aligning boxes

	left	center	right
inline box	`text-align: left;`	`text-align: center;`	`text-align: right;`
block box	`float left;`	`margin: 20px auto; /* vertical horizontal */`	`float right;`

CSS Selectors

Pseudo-classes

type selector	all tags of type `<p>`	`p {}`
class selector	all tags with `class='button'`	`.button {}`
id selector	single element with `id='button-1'`	`#button-1 {}`
descendant selector		`.synopsis em {}`
child selector		`.menu > li {}`

Floats

Flexbox

	default for	forced with
		`display:`
inline box	a, ...	`inline`
block box	div, ...	`block`
flexbox		`flex`

Advanced positioning

	`position:`
static	`static`	CSS box model, floats, flexbox
relative	`relative`
absolute	`absolute`		absolute, referenced to parent element
relatively absolute
fixed	`fixed`		absolute, referenced to browser window; do not scroll

cursor

Responsive design
- Info
  - Responsive Web Design Patterns
    - Mostly fluid
    - Column drop
    - Layout shifter
    - Tiny tweaks
    - Off canvas
- Techniques
  - media queries
    - Using media queries (Mozilla)
    - @media
    - @media only screen and (min-width: 401px) and (max-width: 960px) { ... }
  - disable default zoom out (960px) in mobile browsers (can be re-enabled from browser preferences: "Show desktop version"):
    - Responsive Web Design - The Viewport
    - Quick Tip: Don’t Forget the Viewport Meta Tag
    - <meta name='viewport' content='width=device-width, initial-scale=1.0, maximum-scale=1.0' />
    - check it from desktop with:
      - Chrome: Toggle Device Toolbar
      - Firefox: ...
Responsive images
Semantic HTML
HTML forms
Web Typography

Web Page design for designers
Any Browser
Hints for Web Authors
Style Sheets
Conception de documents WWW
Design and Development Documents
Do's and Don'ts of Web style
W3: HyperText Design Issues
Introduction à la Création de Documents pour le WWW
Le Web à l'EPFL
LUDWEB/Web Design
Yale C/AIM WWW Style Manual
The Six Webs
Alta en buscadores
Vischeck (color blindless simulator)
Mida de la lletra / Font size
- Chrome render different font sizes in mobile browsers
  - Chromium’s Text Autosizer
  - Solució / Solution
    - <meta name="viewport" content="width=device-width">
Color / Colour
- Color Combinations and Color Schemes
DevEdge (Netscape)
- Netscape Gecko Compatibility Handbook
- Updating DHTML Web Pages for next generation browsers
- Diagnosing Web Pages for Netscape 6/Mozilla Support (Netscape vs Explorer appearance)
- DevEdge articles
- Incorrect MIME Type for CSS Files
Developer at Mozilla
- Web developer documentation
- Using Web Standards in Your Web Pages
- Mozilla's DOCTYPE sniffing
Tidy
- Clean up your Web pages with HTML TIDY
- HTML Tidy Project Page
  - Quick Reference
  - HTML tidy service (online)
  - Problemes / Problems
    - els comentaris es mouen (això no passa a la versió per a html5)
- HTML5
  - HTML Tidy for HTML5 (experimental)
    - Source code
    - Quick reference
- Utilització / Usage:
..
Website Design
Dave Raggett (W3C)
- Basic HTML
- Advanced HTML: entitities, tables...

Metadades incrustades / Embedded metadata
- OpenURL ContextObject in SPAN (COinS)
  - COinS generator
  - OpenURL Referrer (Firefox plugin)
  - Used by:
    - refbase (wp)
Testers / Validations
Utilitats / Utilities
- HTML parsing
- html2ps
- TTH: the TEX to HTML translator
- WebTools
- Showstat (page statistics)
- NedStat España (page statistics)
- Link checker
- gospy-applet Project

CSS

Web Style Sheets home page
CSS (All standards and drafts) (W3C)
- CSS1
- CSS2
- CSS3
  - CSS Paged Media Module Level 3
    - @page
- CSS TV Profile
- Tableless layout HOWTO
- W3C Public CVS Repository
  - csswg
    - CSS Paged Media Module Level 3 (css-page)
- Extensions
  - Sass
    - nesting
    - variables
    - ...
  - Compass
    - CSS Authoring framework

	W3 Schools	developer Mozilla
link, visited, hover, focus, after, ...	CSS selectors	Pseudo-classes

CSS usage in Confluence Export PDF:
- Customizing exports to PDF (Confluence 5.1)
- Advanced PDF Export Customisations (Confluence 5.1)
- How can I pull in the page title to the PDF export header?
CSS validator
CSSED - "A GTK-2 CSS Editor"
CSS (wp)
- Comparison of layout engines (Cascading Style Sheets) (wp)
- Tableless web design
Mozilla CSS support chart

selector	usage	example	usage from HTML
	for all elements of specified type	`h1 {...}`	`<h1>`
id selector	for a single, unique element	`#toto {...}`	`<... id="toto">`
class selector	for any type of element, with this class	`.toto {...}`	`<p class="toto"> <h1 class="toto"> ...`
class selector	only for the specified type, with this class	`h1.toto {...}`	`<h1 class="toto">`

Inserció / Insertion
- external
  - toto.css
    - h1 { /* comment */ color:red; text-align:center; }
  - html
    - <head> <link rel="stylesheet" type="text/css" href="toto.css" /> </head>
- internal
  - <head> <style> h1 { color:red; text-align:center; } </style> </head>
- inline
  - <body> <h1 style="color:red; text-align:center;"/> </body>

Posicionament / Positioning

	description	original space preserved (does it affect the position of other elements?) (is on the normal flow?)	move with scroll
static	default	yes	yes
fixed	relative to browser window	no	no
relative	relative to its normal position	yes	yes
absolute	relative to the first parent element that has a position other than static. If no such element is found, the containing block is <html>	no	yes

Margins
- ...
Exemples / Examples
- HbbTV
- mostra el contorn de l'element / Outline:
  - outline: #00dd00 dotted medium;
  - outline-style: dotted; outline-color: #00dd00; outline-width: medium;
- canvi de color d'enllaços visitats
  - a.intern { color: #bb0000; } a.intern:visited { color: #9b6a5b; }
- afegir imatge al final
  - .nou:after { content: url(im/nou.png); }
- tooltips
  - How TO - Tooltip
  - <div class="tooltip">Hover over me <span class="tooltiptext">Tooltip text</span> </div>

XHTML

XML

Extensible Markup Language (XML) (W3C)
Schemas
- DTD
- W3C XML Schema (XSD)
  - equivalent for JSON: JSON Schema
  - XML Schema Tutorial (W3 Schools)
- RelaxNG
The XML library for Gnome
<recursos XML/>
XML Pitstop
- XML Tools

XPath

XPath tutorial (w3 schools)
Free Online XPath Tester / Evaluator

...

	`<root xmlns:foo="http://www.foo.org/" xmlns:bar="http://www.bar.org"> <actors> <actor id="1">Christian Bale</actor> <actor id="2">Liam Neeson</actor> <actor id="3">Michael Caine</actor> </actors> <foo:singers> <foo:singer id="4">Tom Waits</foo:singer> <foo:singer id="5">B.B. King</foo:singer> <foo:singer id="6">Ray Charles</foo:singer> </foo:singers> </root>`
selecciona el node del document	`/`
selecciona el node 'root'	`/root`
selecciona tots els elements 'actor' que són directament fills de l'elelement 'actors'	`/root/actors/actor`
selecciona tots els elements 'singer', siguin on siguin al document	`//foo:singer`
selecciona l'atribut 'id' dels elements 'singer', siguin on siguin	`//foo:singer/@id`
selecciona el valor del text del primer element 'actor'	`//actor[1]/text()`
selecciona l'últim element 'actor'	`//actor[last()]`
selecciona el primer i segon elements 'actor' segons la seva posició	`//actor[position() < 3]`
selecciona tots els elements 'actor' que tinguin atribut 'id'	`//actor[@id]`
selecciona l'element 'actor' que té l'atribut 'id' amb valor '3'	`//actor[@id='3']`
selecciona l'element 'actor' que té l'atribut 'id' amb valor més petit o igual que '3'	`//actor[@id<='3']`
selecciona totsel fills del node 'singers'	`/root/foo:singers/*`
selecciona tots els elements del document	`//*`
selecciona els elements 'actor' i els elements 'singer'	`//actor\|//foo:singer`
selecciona el nom del primer element del document	`name(//**[1])`
selecciona el valor numèric de l'atribut del primer element 'actor'	`number(//actor[1]/@id)`
selecciona la representació en text del valor de l'atribut 'id' del primer element 'actor'	`string(//actor[1]/@id)`
selecciona la longitud del valor textual del primer element 'actor'	`string-length(//actor[1]/text())`
selecciona el nom local del primer element 'singer' (és a dir, sense espais)	`local-name(//foo:singer[1])`
selecciona el nombre d'elements 'singer'	`count(//foo:singer)`
selecciona la suma dels atributs 'id' dels elements 'singer'	`sum(//foo:singer/@id)`

Validate
- Formats
  - DTD, XSD
- Tools
  - xmlstarlet: val
Style
- XSL (EXtensible Stylesheet Language)
  - style sheets for XML (CSS = style sheets for HTML)

Transform

Formats

XSLT

language	desc	tutorial	reference	functions
XSLT (XSL Transformations)	transforming XML documents	XSLT Introduction	XSLT elements reference <xsl:template> <xsl:value-of> <xsl:for-each> <xsl:sort> <xsl:if> <xsl:choose>, <xsl:when>, <xsl:otherwise> <xsl:apply-templates>	Functions reference
XPath	navigating in XML documents	XPath tutorial
XQuery	querying XML documents

XML and XSLT (w3schools)

Conversion XML -> HTML

browser will take file specified on xml-stylesheet to render XML as HTML; if no xml-stylesheet is specified, it will render raw xml

cdcatalog.xml	cdcatalog.xsl (xml -> html)	cdcatalog_text.xsl (xml -> txt)
`<?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="cdcatalog.xsl"?> <catalog> <cd> <title>Empire Burlesque</title> <artist>Bob Dylan</artist> <country>USA</country> <company>Columbia</company> <price>10.90</price> <year>1985</year> </cd> . . </catalog>`	`<?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">` `<xsl:output method="html"/> <xsl:template match="/"> <html> <body> <h2>My CD Collection</h2> <table border="1"> <tr bgcolor="#9acd32"> <th>Title</th> <th>Artist</th> </tr> <xsl:for-each select="catalog/cd">` `<xsl:sort select="artist"/> <tr> <td><xsl:value-of select="title"/></td> <td><xsl:value-of select="artist"/></td> </tr> </xsl:for-each> </table> </body> </html> </xsl:template> </xsl:stylesheet>`	`<?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:output method="text" media-type="text/plain"/> <xsl:template match="/"> <xsl:for-each select="catalog/cd"> <xsl:value-of select="title"/> <xsl:text>: </xsl:text> <xsl:value-of select="artist"/> <xsl:text> </xsl:text> </xsl:for-each> </xsl:template> </xsl:stylesheet>`

Conversion XML -> JSON
- Convert XML to JSON using XSLT (Bizcoder)
- JayDaley/XML-to-JSON-in-XSLT
- doekman/xml2json-xslt

Tools
- xmlstarlet: tr
  - xmlstarlet tr cdcatalog.xsl cdcatalog.xml
  - xmlstarlet tr cdcatalog_text.xsl cdcatalog.xml
- nginx xslt module
Exemples / Examples
- JMRI
- nginx xslt module
  - nginx-rtmp-module stats

Editors
- XML Editing (openlaszlo)
- Emacs
  - XAE
  - modes
    - PSGML
    - nXML (*)
      - schemas must be in RelaxNG format (.rnc) (conversion from XSD)
      - add lines to file .../nxml-mode-20041004/schema/schemas.xml:
        
        <namespace ns="urn:mpeg:mpeg4:SAF:2005" typeId="SAF"/>
        
        <typeId id="SAF" uri="saf2.rnc"/>
      - ~/.emacs
        
        (load ".../nxml-mode-20041004/rng-auto.el")
        (setq auto-mode-alist
        (cons '("\\.\$xml\\|xsl\\|rng\\|xhtml\\|xsr\$\\'" . nxml-mode)
        auto-mode-alist))
- Eclipse
  - Help -> Install new software: search for "XML"
  - XMLBuddy
- qxmledit
- Conglomerate
- QXmlEditor
- KXML Editor (KDE)
  - kxmleditor
- MlView (gnome)
- Commercial
  - XMLSpy
  - <oXygen/>
- From bash:
Biblioteques / Libraries
- Xerces (C++, Java, Perl)
- - DOM programming guide
  - Microsoft Visual C 7.1 (2003) with RTTI (xsd)
  - Trucs
  - Chapter 14: XML (C++ Cookbook)
- libxml2 (C) / libxml++ (C++)
- libwbxml (WBXML)
- irrXML
Compressió / Compression
- BiM (Expway)
Eines / Tools
- Command-line XML processing
- How to parse XML using shellscript? [duplicate]
  - xmllint
  - xmlsh
    - commands
  - sgrep
  - xml-coreutils
- xmlstarlet (transform, query, validate and edit)
  - Documentation (single html)
  - Instal·lació / Installation
    - Mageia
      - urpmi xmlstarlet
      - binary is xmlstartlet instead of xml
  - Changing the value of an XML element depending on the value of another element in the tree
  - Searching for XML tag by value between them and inserting a new tag in shell script
  - Estructura / Structure:
    - xmlstarlet el toto.xml
  - Reformata / Reformat (prettify)
    - xmlstarlet format --encode utf-8 input.xml >output.xml
  - Validating XML documents (using XSD, DTD)
    - xml val -s toto.xsd -e toto.xml >validation_output.txt 2>&1
    - xmlstarlet val -s toto.xsd -e toto.xml >validation_output.txt 2>&1
  - Transforming XML documents (using XSLT)
    - xml tr xsl/param1.xsl xml/table.xml
    - xmlstarlet tr xsl/param1.xsl xml/table.xml
    - set parameters defined in param1.xsl as <xsl:param name="my_param">
      - xmlstarlet tr xsl/param1.xsl -s my_param=my_value xml/table.xml
  - Print to plain text
    - How to convert multi level xml to a single line containing all levels using XMLSTARLET
    - XMLStarlet: Printing one line per item, while using datum from parent element
    - Use XMLStarlet to parse XML in the Linux terminal
    - xmlstarlet sel ...
      - -C - show xsl generated code
      - -T - text output
      - -t - next arguments specify a template
        
        -m, --match <xpath> - match
        
        -v, --value-of <xpath> - print value (between open and closing tags)
        
        -o <string> - print literal string
        
        -n - print newline
        
        ...
    - Exemples / Examples
      - myfile.xml
        
        <?xml version="1.0" encoding="UTF-8" standalone="no"?> <xml> <os> <linux> <distribution when="today"> <name>Fedora</name> <release>7</release> <codename>Moonshine</codename> <clau usage="signing"> clau per signar </clau> <clau usage="encryption"> clau per xifrar </clau> <spins> <name>Live</name> <name> Fedora </name> <name>Everything</name> </spins> </distribution> <distribution when="tomorrow"> <name>Fedora Core</name> <release>6</release> <codename>Zod</codename> <spins></spins> </distribution> </linux> </os> </xml>
      - estructura:
        
        xmlstarlet el input.xml
        
        sense ordre ni repetició:
        
        xmlstarlet el -u input.xml
        
        amb atributs:
        
        xmlstarlet el -a input.xml
        
        amb els valors dels atributs:
        
        xmlstarlet el -v input.xml
      - parts a partir de distribution, sense les etiquetes, substituïdes per espais buits:
        
        xmlstarlet select --template --value-of /xml/os/linux/distribution input.xml
      - si voleu tot l'xml:
        
        xmlstarlet select --template --copy-of /xml/os/linux/distribution input.xml
      - només el contingut de distribution/name:
        
        xmlstarlet select --template --value-of /xml/os/linux/distribution/name --nl myfile.xml
      - selecció condicional, fent servir XPath:
        
        xmlstarlet sel --template --value-of '/xml/os/linux/distribution[name = "Fedora Core"]/release' --nl myfile.xml
        
        xmlstarlet sel --template --value-of '/xml/os/linux/distribution[name = "Fedora Core"]/codename' --nl myfile.xml
        
        filtre per atribut, amb sortida xml (--copy-of):
        
        xmlstarlet sel --template --match '/xml/os/linux/distribution/clau[@usage = "encryption"]' --copy-of '.' --nl myfile.xml
      - match seguit de value-of:
        
        xmlstarlet sel --template --match '/xml/os/linux/distribution/spins' --value-of '.' --nl myfile.xml
        
        xmlstarlet sel --template --match '/xml/os/linux/distribution' --value-of 'spins' --nl myfile.xml
      - atribut:
        
        xmlstarlet sel --template --match '/xml/os/linux/distribution' --value-of '@when' --nl myfile.xml
      - si voleu eliminar els espais i canvis de línia:
        
        xmlstarlet sel --template --match '/xml/os/linux/distribution/spins/name' --value-of 'normalize-space()' --nl myfile.xml
        
        xmlstarlet sel --noblanks --template --match '/xml/os/linux/distribution/clau[@usage = "encryption"]' --value-of 'normalize-space()' myfile.xml
      - si el fitxer especifica xmlns (namespace):
        
        myfile_ns.xml
        
        <?xml version="1.0" encoding="UTF-8" standalone="no"?> <xml xmlns="..."> <os> <linux> <distribution when="today"> <name>Fedora</name> <release>7</release> <codename>Moonshine</codename> <spins> <name>Live</name> <name>Fedora</name> <name>Everything</name> </spins> </distribution> <distribution when="tomorrow"> <name>Fedora Core</name> <release>6</release> <codename>Zod</codename> <spins></spins> </distribution> </linux> </os> </xml>
        
        xmlstarlet sel --template --match '//_:xml' --value-of '_:os' --nl myfile_ns.xml
        
        xmlstarlet sel --template --match '//_:xml' --value-of '_:os/_:linux/_:distribution/_:name' --nl myfile_ns.xml
        
        SAML
        
        xmlstarlet sel --template --match '//_:EntityDescriptor/_:AttributeAuthorityDescriptor/_:KeyDescriptor[@use="signing"][1]/ds:KeyInfo/ds:X509Data/ds:X509Certificate' --value-of 'normalize-space()' metadata.xml
        
        echo -e "-----BEGIN CERTIFICATE-----$(echo "$(xmlstarlet sel --template --match '//_:EntityDescriptor/_:AttributeAuthorityDescriptor/_:KeyDescriptor[@use="signing"][2]/ds:KeyInfo/ds:X509Data/ds:X509Certificate' --value-of '.' metadata.xml)" | tr -d ' ')\n-----END CERTIFICATE-----" >cert_5.txt
        
        (no acaba de funcionar) xmlstarlet sel --template --var nl -n -b --match '//_:EntityDescriptor/_:AttributeAuthorityDescriptor/_:KeyDescriptor[@use="signing"][2]/ds:KeyInfo/ds:X509Data' -o "-----BEGIN CERTIFICATE-----" --value-of 'concat(ds:X509Certificate,$nl,"-----END CERTIFICATE-----")' --nl metadata.xml
      - ...
    - Additional functions from exslt (can be used inside -v):
      - concat()
      - str:replace() (fn:replace) (fn:replace from XQuery; csv.xqy)
        
        Replace part of URL in XML with xmlstarlet
        
        ...
      - ...
    - print to csv
      - info
        
        csv needs double quotes to be escaped (doubled)
        
        double quotes using xslt
        
        how to replace one double quotes with two double quotes in XSLT
      - passos / steps
        
        get xsl code (-C) by running: xmlstarlet sel -C ... >my_xml_to_csv.xsl
        
        edit the bottom part of my_xml_to_csv.xsl, to replace template value-of-template:
        
        <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:exslt="http://exslt.org/common" version="1.0" extension-element-prefixes="exslt"> ... <xsl:template name="value-of-template-old"> <xsl:param name="select"/> <xsl:value-of select="$select"/> <xsl:for-each select="exslt:node-set($select)[position()>1]"> <xsl:value-of select="'
        '"/> <xsl:value-of select="."/> </xsl:for-each> </xsl:template> <xsl:variable name="quot">"</xsl:variable> <xsl:variable name="doublequot">""</xsl:variable> <xsl:template name="value-of-template"> <xsl:param name="select" /> <xsl:choose> <xsl:when test="contains($select, $quot)"> <xsl:value-of select="substring-before($select,$quot)" /> <xsl:value-of select="$doublequot" /> <xsl:call-template name="value-of-template"> <xsl:with-param name="select" select="substring-after($select,$quot)" /> </xsl:call-template> </xsl:when> <xsl:otherwise> <xsl:value-of select="$select" /> </xsl:otherwise> </xsl:choose> </xsl:template>
        
        </xsl:stylesheet>
        
        ...
        
        run xmlstarlet tr with the xsl file:
        
        xmlstarlet tr my_xml_to_csv.xsl toto.xml >toto.csv
        
        you may want to convert from csv to xlsx
    - ...
  - ...
- xmlindent
- Xalan command line utility (Apache)
- Sun Multi-Schema Validator (MSV)
  - rngconv: conversion (.xsd,.dtd) -> .rng (RelaxNG XML syntax)
    - java -jar rngconv.jar toto.xsd > toto.rng
    - java -jar rngconv.jar -dtd toto.dtd > toto.rng
- Trang (RelaxNG conversion: .rng (XML syntax) -> .rnc(compact syntax))
  - java -jar trang.jar toto.rng toto.rnc
Trucs / Tips
- Dealing with Markup Characters, Part II: CDATA sections (usage of CDATA to avoid parsing inside scripts)
  - <script>//<![CDATA[
  - //]]> </script>
- XercesDOMParser Supported Features
  - setDoValidation
Diferències / Differences (text diff)
- General
  - Are there any free Xml Diff/Merge tools available? [closed]
  - Open Source XML Diff written in Java
  - Stackoverflow: xml+diff
    - XML Diff and Merge (Java)
- RFC 5261
  - xmlpatch
- Eines / Tools
  - xmldiff (Python, GPL)
  - diffxml / patchxml (GPL)
    - Dissertation
    - DUL Specification
  - DiffX (Java)
  - xmlpatch (C) (implements RFC 5261)
  - VMTools (Java)
  - OpenSHORE XML Merger
  - xxdiff

JavaScript

ECMAScript
- Wikipedia
Documentation
- JavaScript language overview
- Event reference (MDN)
- Web technology for developers
  - JavaScript
    - JavaScript technologies overview
    - JavaScript Guide (Mozilla Developer Network)
    - A re-introduction to JavaScript (Mozilla)
- Web API reference
  - Document Object Model
  - ...

Equivalències

	JavaScript	Python
Data structures	`undefined`: an identifier, absence of a value `null`: a keyword, absence of an object
Objects	Definition of an object:`const obj = { name: "Carrot", for: "Max", details: { color: "orange", size: 12, }, };` Access to its properties: obj.name obj["name"]	Definition of a dictionary:`obj = { "name": "Carrot", "for": "Max", "details": { "color": "orange", "size": 12, } }` Access to its properties: `obj["name"]`
Arrays	`for (const i of mylist)`	`for i in mylist`
Functions	`...args`	`*args`
	Basic function / Function declaration: `function add(x, y) { const total = x + y; return total; }`
	Anonymous function / Function expression: `const myvar = function (...args) { // ... } // Traditional anonymous function (function (a) { return a + 100; });`
	Anonymous arrow function (word `function` is replaced by an arrow):`const myvar = (...args) => { // ... } // 1. Remove the word "function" and place arrow between the argument and opening body brace (a) => { return a + 100; }; // 2. Remove the body braces and word "return" — the return is implied. (a) => a + 100; // 3. Remove the parameter parentheses a => a + 100;`	Lambda: ...
	Immediately invoked function expression (IIFE):`(function () { // ... })();`	Lambda IIEF:`(lambda x, y: x + y)(2, 3)`
Classes	Class declaration:`class MyClass { constructor(...) { // ... #my_private_field ... static my_static_method ... } } // Declaration class Rectangle { constructor(height, width) { this.height = height; this.width = width; } }`	`class MyClass: def __init__(self, ...): ... _my_private_method ...`
	Class expression: `// Expression; the class is anonymous but assigned to a variable const Rectangle = class { constructor(height, width) { this.height = height; this.width = width; } }; // Expression; the class has its own name const Rectangle = class Rectangle2 { constructor(height, width) { this.height = height; this.width = width; } };`
	Inheritance: `class Dog extends Animal { ... }`	`class Dog(Animal):`
Asynchronous	Callback-based: `fs.readFile(filename, (err, content) => { // This callback is invoked when the file is read, which could be after a while if (err) { throw err; } console.log(content); }); // Code here will be executed while the file is waiting to be read`
	Promise-based (chained promises: `.then().catch()`): `fs.readFile(filename) .then((content) => { // What to do when the file is read console.log(content); }) .catch((err) => { throw err; }); // Code here will be executed while the file is waiting to be read`
	async/await: `async function readFile(filename) { const content = await fs.readFile(filename); console.log(content); }`	asyncio ...
Modules	`import { foo } from "./foo.js";`	`import foo from foo import bar`

Tutorials
- Getting started with the Web (MDN)
  - JavaScript basics
Reference
- HTML DOM Event Object
- match() Method
JavaScript Kit
- Tackling JavaScript strict warnings (from Netscape 6 and Mozilla)
- Overview of scripting in NS6
JavaScript Gamelan Directory
DevEdge (Netscape)
- JavaScript Developer Central
- Client-Side JavaScript Guide
- Client-Side JavaScript Reference
- Core JavaScript Guide
- Netscape JavaScript Debugger
Sash
- Sash: the Javascript runtime
- SashXB for Linux
JavaScript Calendar
Free Cut-and-Paste JavaScript
Javascript PC emulator (micro Linux)

JSON

Comparison to XML

XML	JSON
XSD, DTD	JSON Schema
XPath	JSON Pointer (RFC 6901) JMESPath
XSLT	XSLT equivalent for JSON

Conversion XML/JSON
- Freeformatter
  - JSON to XML Converter
  - XML to JSON Converter

JavaScript Object Notation JSON
- To XML and Back: Using JSON in Android (Linux Magazine)
Search
- JMESPath

JSON Schema

Similar to xsd for xml
Documents (IETF)
- JSON Schema Validation: A Vocabulary for Structural Validation of JSON (IETF draft-wright-json-schema-validation-00)
- JSON Schema: A Media Type for Describing JSON Documents (IETF draft-wright-json-schema-00)
Aprenentatge / Learning
- Understanding JSON Schema
  - Structuring a complex schema
    - JSON Pointer
Extensió / Extension
- JSON Hyper-Schema
  - Getting started with JSON Hyper-Schema

Eines / Tools

Implementations
JSON Schema Tool
- automatically create a JSON Schema from a JSON example
Validation
- JSON Schema Lint :: JSON Schema Validator

Packages

usage	package	language	JSON Schema draft version	demo
form creation	Angular JSON Schema Form	Angular 4	6	Angular JSON Schema Form — Demonstration Playground
	json-editor	Javascript	5	Interactive demo Interactive Playground
	django-jsonforms	Django
validation	jsonschema	Python

json-editor
- json schema -> form -> json
- field ordering
  - Property Ordering
  - "propertyOrder": 1000
- i18n for labels
- enum values from external document
  - What is the best approach to populate enum values from a remote source #288
Django
- JSON schema validation with Django Rest Framework

used by:
- OpenAPI Initiative (OAI) (Swagger)
  - "OpenAPI 3.0 uses an extended subset of JSON Schema Specification Wright Draft 00 (aka Draft 5) to describe the data formats."
- Django Rest Framework: Schemas
Exemples / Examples
- JSON Schema Store
- paypal/api-standards/v1/schema/json (PayPal) (draft-04)
- simple.schema.json
  - { "$schema": "http://json-schema.org/draft-07/schema#", "definitions": {}, "type": "object", "properties": { "name": { "title": "Name", "type": "string" }, "label": { "title": "Label", "type": "string", "enum": ["abc","zxc"], "options": { "enum_titles": ["Abc","Zxc"] } } }, "required": [ "name", "label" ] }

JSON-LD (Linking Data)
- Exemples / Examples
  - IPTC (International Press Telecommunications Council)
    - Developer Site
      - SubjectCode: cptall-es.json
        
        extract...
      - MediaTopic: cptall-x-all.json
  - schema.org
    - used by:
      - Google: Understand how structured data works
        
        index.html
        
        <script type="application/ld+json"> { "@context": "https://schema.org", "@type": "Organization", "url": "http://www.example.com", "name": "Unlimited Ball Bearings Corp.", "contactPoint": { "@type": "ContactPoint", "telephone": "+1-401-555-1212", "contactType": "Customer service" } } </script>

Parsers

Parsing json with sed and awk
How to parse JSON string via command line on Linux (jq)
Python
- Django

command line

jsawk
- resty (script wrapper for curl)

jq play (interactive player)
removal of nodes, by naming nodes to excise #37
- del()

Ús / Usage

Tutorial
Manual
Cookbook
jq [options] '<commands>' file.json
jq -f file.jq ...
- file.jq
  - <sequential_command_1> | <sequential_command_2> | (<parallel_command_3.1>), (<parallel_command_3.2>) | ...
cat file.json | jq [options] '<commands>'
echo $my_var | jq [options] '<commands>'

special character	description
`\|`	separate chained commands
`()`	group commands
`,`	separate parallel commands
+	concatenate strings

Jq to replace text directly on file (like sed -i)
- jq '...' toto.json

Exemples / Examples
- AWS
- mostra el contingut del fitxer / show the file content
  - jq '.' toto.json | sponge toto.json
- mostra el contingut de la variable / show the variable content
  - echo $my_var | jq '.'
- multiple input files
  - show file1
    - jq -s '.[0]' file1.json file2.json
  - show file2
    - jq -s '.[1]' file1.json file2.json
  - merge file1 and file2:
    - How to merge 2 json file using jq?
    - merging many json files into one
      - inner list
        
        AWS cloudwatch: awscloud merge of config files
    - jq -s '.[0] * .[1]' file1.json file2.json
- mostra una línia per a cada entrada, amb dos dels camps separats per un espai:
  - jq -r '.[] | [.field_1, .field_2] | join(" ")'
- posa cada element d'una llista en una sola línia / put each element of an array into a single line
  - toto.json
    - { "llista": [ { "nom": "primer", "valor": 1, "preu": 11 }, { "nom": "segon", "valor": 2, "preu": 22 } ] }
  - jq -r '.llista[] | [.nom, .valor | tostring ] | join(" ")' toto.json
    - primer 1
      segon 2
- crea un array amb els valors d'un sol dels camps:
  - jq '[.llista[] | .nom]' toto.json
- posa en una sola línia, un sol camp per a cada entrada:
  - jq '[.llista[] | .nom] | join(" ")' toto.json
  - ordenats:
    - jq '[.llista[] | .nom] | sort | join(" ")' toto.json
  - jq '.llista[] | .nom' toto.json | paste -sd' '
    - "primer" "segon"
  - es pot posar en un array de bash:
    - declare -a my_array
    - (?) my_array=($(echo $my_json | jq '.llista[] | []'jq '.llista[] | .nom' toto.json | paste -sd' '))
    - my_array=($(echo $my_json | jq '.llista[] | .nom' | paste -sd' '))
    - my_array=($(jq '.llista[] | .nom' toto.json | paste -sd' '))
- posa en una sola línia la clau i el seu valor:
  - jq 'to_entries[] | [.key, .value] | join(" ")' toto.json
  - i es posa en un array associatiu de bash:
    - #!/bin/bash input="{ \"el primer\": \"un u\", \"el segon\": \"un dos\", \"el tercer\": \"un tres\" }" declare -A my_array while IFS= read -r element do echo "-- read element from jq: $element" key=$(echo "$element" | awk 'BEGIN{FS="#"} {print $1}') value=$(echo "$element" | awk 'BEGIN{FS="#"} {print $2}') my_array["$key"]="$value" done < <(echo "$input" | jq -r 'to_entries[] | [.key, .value] | join("#")') for key in "${!my_array[@]}" do echo "${key}: ${my_array[$key]}" done exit 0
- create json
  - jq -n '{foo:"bar",foo2:3}'
  - jq -n '.foo="bar"'
  - jq -n '.[0].foo=1'
- variables bash
  - Using Variables in shell script #626
  - nom="el meu nom" valor=123 json_code=$(jq -n -c "{nom:\"$nom\",valor:$valor}")
- Afegeix entrades a una llista / Add entries to a list
  - part_1=$(jq -n '[{foo:"bar",foo2:3}]') part_2=$(jq -n '[{foo:"bar2",foo2:4}]') part_1=$(echo $part_1 | jq ". += $part_2") echo $part_1 | jq '.'
- Substitució / Replacement
  - Update-assignment: |=
- Manipulate json as a bash variable:
  - body=$(jq -n '{foo:"bar",foo2:3}') echo "${body}" | jq '.' body=$(echo "${body}" | jq '. += {foo3:"bar3"}') echo "${body}" | jq '.' value=bar4 body=$(echo "${body}" | jq ". += {foo3:\"${value}\"}") echo "${body}" | jq '.'
    
    object=$(jq -cr -n '{foo4:"bar4",foo5:5}') echo "${object}" | jq '.' body=$(echo "${body}" | jq ". += {foo6:${object}}") echo "${body}" | jq '.'
- Subconjunt / Subset
  - jq '{field1:.field1,field2:.field2}'
- comprova si existeix my_key / check if my_key exists
  - if has("my_key") then .my_key else empty end
  - # desaconsellat, perquè si la clau existeix is és un booleà fals es complirà l'if i posarà empty my_value=$(cat toto.json | jq -r 'if .my_key then .my_key else empty end') if ! [ "$my_value" ] then exit 1 fi
- if "my_key" exists, output "-my_key <my_key_value>"
  - (.my_key | values | tostring | "-my_key "+.),
- fallback (alternative operator)
  - si el valor és null, retorna una cadena buida
    - my_value=$(echo ${curl_response} | jq '.my_key // empty')
  - si el valor és null, retorna una cadena emmagatzemada en una variable de la shell:
    - my_fallback_value="toto"
      my_value=$(echo ${curl_response} | jq ".my_key // ${my_fallback_value}")
- filters
  - @csv
  - @tsv
- map
  - map(x) is the same as:
    - [.[] | x]
- find / select
  - ".nginx.applications[] | select(.name ==\"$application_name\") | .record"
  - Només la primera / Only the first occurrence:
    - ".nginx.applications[] | map( select(.name ==\"$application_name\") | .record) | if .[0] then .[0] else empty end"
  - select elements with name="my_prefix...":
    - '.my_list[] | select(.name | startswith("my_prefix") )'
  - How to filter an array of objects based on values in an inner array with jq?
  - boolean
    - select(... and ...)
    - select(... or ...)
  - key and value
    - Select objects based on value of variable in object using jq
    - Filter objects based on tags in an array
    - given the following json:
      - [ { "title": "first", "number": 1, "tags": [ {"key": "foo1","value": "bar1"}, {"key": "foo2","value": "bar2"} ] }, { "title": "second", "number": 2, "tags": [ {"key": "foo1","value": "bar2"}, {"key": "foo2","value": "bar1"} ] }, { "title": "third", "number": 3, "tags": [ {"key": "foo1","value": "bar1"} ] } ]
      - select elements with a tag foo1/bar1 (first, third, but not second):
        
        jq '.[] | select( .tags[] | . and .key=="foo1" and .value=="bar1")'
  - remove empty elements
    - | select( length>0 )
- Shellscript: loop through a JSON array with jq
  - while IFS= read -r element do echo "-- read element from jq: $element" done < <(echo "$input" | jq -c '.toto[]')
  - while IFS= read -r; do echo "read content from jq: $REPLY" done < <(echo "$input" | jq -c '.toto[]')
  - while IFS= read -r element do echo "-- read element from jq: $element" done < <(jq -c '.toto[]' $filename)
- sort (descending) by sum of several fields
  - [ .[] | ([.components[] | .bandwidth] | add ) as $total_bandwidth | .+{tbw:$total_bandwidth} ] | sort_by(-.tbw)
- sort and swap first and second elements of an array
  - 'sort_by(-.field_1) | .[1] as $second | .[1] = .[0] | .[0] = $second | .[]'
- counter
  - How do I create an incremental index with jq
  - 2 solutions:
    - foreach
      - ...
    - reduce
      - ...
- aplana la jerarquia de claus / flatten key hierarchy
  - Flattening the entire JSON object
  - Flatten nested JSON using jq
  - jq '. | [paths(scalars) as $path | {"key": $path | join("."), "value": getpath($path)}] | from_entries' toto.json
  - jq '. | paths(scalars) as $path | $path | join(".")'
- convert json to ffmpeg parameters
  - .[] | [ ("-vsync vfr"), (.components[] | if .type!="image" then (if .type=="video" then "v" else "a" end) as $tipus | #.lang "-c:"+$tipus, .codec, (if .codec=="aac" then "-strict -2" else empty end), (.preset | values | if (. | length) > 0 then "-preset "+. else empty end), #(.preset | values | "-"+$tipus+"pre "+.), "-b:"+$tipus, (.bandwidth | tostring), (.channels | values | tostring | "-ac "+.), (if .width then "-vf scale=w="+(.width | tostring)+":h="+(.height | tostring) else empty end), #(if .width then "-vf scale=w="+(.width | tostring)+":h="+(.height | tostring)+":force_original_aspect_ratio=decrease" else empty end), #(if .width then "-vf scale='-2:ceil(min(1\\,min("+(.width | tostring)+"/iw\\,"+(.height | tostring)+"/ih))*ih)'" else empty end), (.gop | values | tostring | "-g "+.), (.profile | values | if (. | length) > 0 then "-profile:"+$tipus+" "+. else empty end), #(.profile | values | if (. | length) > 0 then "-profile "+. else empty end), (.level | values | tostring | "-level "+.), (.keyint_min | values | tostring | "-keyint_min "+.), (.sc_threshold | values | tostring | "-sc_threshold "+.) else empty end ), (.segment_duration | tostring | "-hls_time "+.), (.segment_list_size | tostring | "-hls_list_size "+.), (.segment_wrap | tostring | "-hls_wrap "+.), (.segment_start_number | tostring | "-start_number "+.), .name + ".m3u8" ] | join(" ")

Print
- cat toto.json | python -m json.tool
Documentation
- Docson
Conversion
- Typson (TypeScript -> json-schema)

Debugging Javascript
- Javascript debugger (Firefox extension)
JSONP (wp)
Observers
- MutationObserver
Biblioteques / Libraries
- knockout (bindings)

Toolkits, Frameworks

Comparison of Javascript frameworks
CAAT (multi-instance director-based scene-graph manager) (animacions 2D)
APE project (Ajax Push Engine)
mootools
jQuery (wp)
- 80 Useful (Free) jQuery Plugins to Enrich Your Site’s User Experience
- Tablesorter
- plugins
  - gridster.js
- dialog
  - jquery-ui-dialog
- API
  - $.ajax
    - jQuery ajax() Method (w3schools)

Node.js

Getting started
- How much JavaScript do you need to know to use Node.js?

	name	info	installation	usage
Node.js		Installation of Node.js on system	System: Install NodeJS (Angular JS 1: Tutorial) Mageia `urpmi nodejs` (also installs npm) CentOS `sudo yum install nodejs` Debian / Ubuntu `sudo apt-get install nodejs-legacy`
Node.js	nvm	Node.js version management: installation of several versions of Node.js on user home (`~/.nvm/`)	no need to install Node.js on system `npm install -g nvm` (NVM in npm is not the right one? #304) Installing and updating `curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh \| bash` `curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.39.1/install.sh \| bash` (logout and login) `source ~/.bashrc` using Fabric `c.run("curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh \| bash")` `c.run("source ~/.nvm/nvm.sh && nvm install 20")`	see Janus (WebRTC) install and register the latest version of Node.js `nvm install node` install and register a specific version of Node.js `nvm install 14.7` set an alias for default `nvm alias default 14.7` switch to a registered version `nvm use default` switch to system version `nvm use system` show current version `nvm current` list installed versions `nvm ls` search for a specific version `nvm which 14.7`
Package manager	npm		Mageia `urpmi npm` (no needed if installed nodejs) CentOS `sudo yum install npm` Debian / Ubuntu `sudo apt-get install npm` `npm -g install npm@latest`	from `package.json`, install locally to `node_modules/` `npm install` registered package (and install globally: -g, to `/usr/lib/node_modules/ or ~/.nvm/versions/node/vx.y.z/lib/node_modules/`) `npm install -g angular-cli` `[sudo] npm install -g ...` install locally (no -g) in `node_modules/` and register to package.json (--save): `npm install ... -save` list of locally (upwards recursivity) installed packages `npm list` list of globally installed packages `npm -g list`
Packages	angular-cli	Angular	`npm install -g angular-cli`	Used by Eclipse plugin: Angular2 Eclipse
	Bower	packages for deployment manager frameworks libraries assets utilities	`npm install -g bower`	from `bower.json`, install to `your_project/app/bower_components/` `bower install` registered package `bower install jquery`
	CoffeeScript	little language that compiles into JavaScript	`npm install -g coffee-script`
	Grunt	Javascript task runner		grunt-cli
	http-server	HTTP server	`npm install -g http-server`	see Janus (WebRTC)

npm
- Instal·lació i ús / Installation and usage
- Problemes / Problems
  - $ npm npm update check failed
    - Solució / Solution
      - check if you have disk space:
        
        df -h

Angular (Google)
ember.js (wp)
qooxdoo (wp)

Video players (HTML5)
- Video.js
Editors
- Eclipse
  - Angular
- Emacs modes
  - js-mode (23.2)
Packages
- Timeline
Beautifier
- Beautify your JavaScript in Emacs
- js-beautify
  - Online Javascript beautifier
  - Instal·lació / Installation
    - pip
      - pip install jsbeautify
    - git
      - git clone https://github.com/beautify-web/js-beautify.git
  - usage:
    - js-beautify toto.js

Exemples / Examples

Des d'HTML / From HTML
<script type="text/javascript" src="js/toto.js"></script>
<script type='text/javascript' language='javascript'> ... </script>
background colour for "a":
- <a onfocus="this.style.background='yellow'" onblur="this.style.background=''">
- or, in css:
  - a:focus {background-color: yellow;}
redirection
- document.location.href = http://...

location

Location object (w3schools)
Parsing URLs

location: http://example.com:8000/toto1/toto2.html#part2

	value
location.href	http://example.com:8000/toto1/toto2.html#part2
location.protocol	http:
location.host	example.com:8000
location.hostname	example.com
location.port	8000
location.pathname	/toto1/toto2.html#part2 (?)
location.hash	#part2

relative href to a different port:

Relative URL to a different port number in a hyperlink?
http://192.168.1.10:8080/index.html

// To deal with urls that imply a change of port, on the same server.
// They begin with ":".
document.addEventListener('click', function(event) {
  var target = event.target;
  if (target.tagName.toLowerCase() == 'a')
  {
    var parts = target.getAttribute('href').match(/^:(\d+)(.*)/);
    if (parts)
    {
      var adreca_aboluta = location.protocol+"//"+location.hostname+":"+parts[1]+parts[2];
      target.setAttribute('href', adreca_absoluta);
    }
  }
}, false);

<body>
...
<a href=":8000/otherpage.html">Other page on the other port at the same server</a>
...
</body>

In this case, before changing the values, they are the following:
- target="http://192.168.1.10:8080/:8000"
- target.port="8080"
- target.getAttribute('href')=":8000"

Simple output:
- <p>Here is the port: <script type="text/javascript"> document.write(location.port); </script> </p>
Alert pop-up:
- alert("Target port is: "+target.port);
Debug information:
- it appears on Firebug: Console / debug info
- Javascript: Write Console.debug() output to browser?
- console.debug("toto_var is: ", TOTO_VAR);
- console.log("missatge");
Emmagatzematge / Storage
- Galetes / Cookies
- Emmagatzematge DOM / DOM storage
Data / Date
- toLocaleTimeString()
- toISOString()
Misc
- Konami code
  - How to add konami code in a website based on html?

TypeScript

Editors
- Eclipse
  - ...
- typescript-tools
  - Installation
    - npm install -g clausreinke/typescript-tools
  - Editors
    - Emacs: emacs-tss
      - Installation: MELPA

DOM (Document Object Model)

W3C
Getting Ready for the W3C DOM (netscape developer)
DOM Central (netscape devedge)
Dynamic Content with DOM-2 (O'Reilly)
Emmagatzematge / Storage
- DOM Storage (Mozilla)

Dart

Java

JavaFX
Authentication
- Use Java 1.2's Authenticator class
- URL authentication revisited

Gràfics vectorials / Vectorial graphics

	version	profile		features						subsets	software			used by	samples
				scripting	audio	video	µDOM	SMIL			implementation	authoring tools	players
				scripting	audio	video	µDOM	sync	animation		implementation	authoring tools	players
SVG	1.1	full		-	-	-	-					Flame Project? (compilation)	Firefox > 1.5 (subset) SmilScript
		mobile	tiny (cellphone)							Laser v1?	GPAC		TinyLine		TinyLine
		mobile	basic (PDA)								GPAC
	1.2	full		x	x	x	x	x	x
	1.2	mobile	tiny	x	x	x	x	x	x	Laser v2	GPAC		GPAC-based Osmo4 MP4Client Ikivo (phones) Bitflash	MMS 1.3

Utilitats / Utilities (embed into HTML,...)
- svgo (optimizer)
W3Schools: SVG
carto.net SVG tutorial, example and demonstration site
SVG in Firefox
SVG Authoring Guidelines
SVG Implementation and Resource Directory
Scale-a-vector
MIME type:
- image/svg+xml (SVG wiki: MIME type)
SVG 1.2 Tiny Test Suite Implementation Matrix
SVG animation (wp)
- Types
  - Scripting
  - Styling
  - SMIL
- Animation with Inkscape
- MPEG LaSeR / SAF (MPEG-4 part 20)

Unitats / Units

SVG Units (w3.org)
Units in Inkscape (wiki)
Understanding SVG Coordinate Systems and Transformations (Part 1) — The viewport, viewBox, and preserveAspectRatio
Definitions
- User Unit
- Unit Identifier
  - absolute: mm, cm, in, pt, pc, px
  - relative: em, ex, %
- SVG Scale Factor

	desc	examples	python svgwrite
viewport	viewing area; areas outside the viewport are cropped and not visible values are specified as real values (include a unit identifier; if not specified, assumed px): 10mm, 2in, ...	<svg width="10in" height="6in" ...>	import svgwrite from svgwrite import in svg_width = 10 svg_height = 6 dwg = svgwrite.Drawing("toto.svg", profile='full', size=(svg_widthin, svg_heightin))
viewBox	x: from left to right y: from top to bottom parameters are specified in user units (no unit identifier) SVG scale factor is calculated as: 10in/50user_units = 0.2in/user_unit	<svg ... viewBox = "0 0 50 30"	dwg.viewbox(0, 0, svg_width, svg_height)
elements	unitless values are considered as specified in user_units

Creació / Creation
Conversió / Conversion
- inkscape -z -e toto.png -w <width> [-h <height>] toto.svg
- Conversió d'svg a imatge amb Python / Convert svg to image using Python

WMF/EMF (MS Windows)
Programari / Software
- Inkscape
- Editors that Support SVG (SVGi)
- OpenVG
- Flame Project (SVG animations)
- SMIL authoring
- Conversió / Conversion
  - Uniconvertor

Adobe Flash

Test
- Flash Player Help

Install

navegador / browser	paquet del sistema / system package				comprovació de la instal·lació / installation check	activació / activation	notes
navegador / browser	repo	package filename	package name	provided file	comprovació de la instal·lació / installation check	activació / activation	notes
Chrome	-	-	-	-	chrome://components/	chrome://settings/content/flash	Chrome 61: Si s'activa l'opció «Pregunta-m'ho abans» (que no preguntarà mai), s'activa la llista blanca («Permet») i només aquells llocs que hi estiguin llistats tindran permís per a executar Flash.
Firefox	Adobe Flash Player	`flash-player-npapi-27.0.0.130-release.x86_64.rpm`	flash-plugin - Adobe Flash Player NPAPI	`/usr/lib64/mozilla/plugins/libflashplayer.so -> /usr/lib64/flash-plugin/libflashplayer.so`	about:addons Shockwave Flash

Programari / Software
- f4l (Flash animation editor)
- gnash
- swfdec
  - swfdec-gnome (standalone player)
- swftools
Dispositius mòbils / Mobile devices

Altres llenguatges / Other languages (parsers)

W3 Schools
SMIL (Synchronized Multimedia Integration Language) (wp)
- SVG animation
- A-SMIL (SMIL for Digital Signage)
- Authoring
- Players
Biblioteques / Libraries
- libwww (w3c)
Compaq's Web Language (WebL)
World Wide Web Wrapper Factory (W4F) (old)
Website Meta Language (WML)
Ruby/Amazon

Llenguatges de marques / Markup languages

Comparison of document markup languages (wp)
Markdown versus ReStructuredText

	markdown	asciidoc (Gitlab)	rst
headers	`# Heading 1 [[_TOC_]] ## Heading 2 ### Heading 3`	`= Document title :toc: :toclevels: 4 :sectnums: == First section level 1 === First section level 2 == Second section level 1 === Second section level 2`	`###### Part 1 ###### ******* Chapter 1 ******* Section 1 ========= Subsection 1 ------------ Subsubsection 1 ^^^^^^^^^^^^^^^ Parapgraph 1 """"""""""""`
table of contents	`[[_TOC_]]`	`:toc::toclevels: 4`	`.. toctree:: :maxdepth: 2 :caption: Contents: another_rst_file_name_without_extension`
unordered list	`* primer * segon * segon_primer * segon_segon`	`Una llista: * primer * segon segon_primer segon_segon`
ordered list	`1. primer 2. segon`	`. primer . segon`
comentari		`// un comentari`
negreta			`text en negreta`
cursiva			`text en cursiva*`
format	`~barrat~`	`[.line-through]#barrat#`
inline code format	`some code`	`some code`
unicode character		`🗘`
icons	`:white_check_mark:`	`✅` :white_check_mark: ✅ ... {white_check_mark} ...
		admonition types: `NOTE:` `TIP:` `IMPORTANT:` `CAUTION:` `WARNING:`
blocks		`---- listing code ---- .... literal .... ____ Quote ____ ===== Example ===== ** Sidebar **`	`.. note:: My note.`
code block	```python import so ```	`[source, python] ---- import so ----`	`.. code-block:: python import so`
language code block			.. py:function:: lumache.get_random_ingredients(kind=None) Return a list of random ingredients as strings. :param kind: Optional "kind" of ingredients. :type kind: list[str] or None :return: The ingredients list. :rtype: list[str] Otherwise, :py:func:`lumache.get_random_ingredients` will raise an exception.
doctests (code snippets included in documentation that are executed when the documentation is built)			doctests conf.py `import pathlib import sys sys.path.insert(0, pathlib.Path(__file__).parents[2].resolve().as_posix())` `extensions = ["sphinx.ext.doctest"]` my_doc.srt `>>> lumache.get_random_ingredients() expected_result` `make doctest` `make html`
autodoc			Automatic documentation generation from code (using sphinx.ext.autodoc and docstrings in your code) `.. autofunction:: lumache.get_random_ingredients .. autoexception:: lumache.InvalidKindError`
autosummary			(using sphinx.ext.autosummary and docstrings in your code) `.. autosummary:: :toctree: my_subdir_where_rst_will_be_automatically_created_by_... my_first_module`
block inside a list item		`. primer + [source, bash] ---- cd toto ---- . segon + [source, bash] ---- cd tata ----`
local svg image	`![My diagram](/docs/my_diagram.svg)`	`image::/docs/my_diagram.svg[My diagram]`
graphics	```mermaid ```	`[mermaid] .... ....`
link to other section (default)		`See <<_second_section>> == Second section ...`
link to other section (custom link and text)		`See <<toto,Toto section>>` `[#toto] == Second section ...`	See :ref:`Toto section <toto>` .. _toto: Second section -------------- ...
link to other page		`link:my_other_page.adoc[My Other Page]`	:doc:`my_other_document`
link to external url	`[My Site](https://mysite.org)`	`https://mysite.org/[My Site]`	`My Site <https://mysite.org/>`_
table	`\|primera\|segona\| \|-------\|------\| \|1 \|2 \| \|3 \|4 \|`	`.Table title [%autowidth] \|=== \|primera\|segona \|1 \|2 \|3 \|4 \|===`	(Emacs: table.el)
2 línies de capçaleres		\|`=== \| primera \| segona h\|tercera h\|quarta \| 1 \| 2 \|===`
colspan		`3+\|This cell spans 3 cols`
rowspan		`.2+\|This cell spans 2 rows`
list inside a cell		`[cols="d,a"] \|=== \|primera\|segona \|1 \|2 * primer * segon \|3 \|4 * tercer * quart \|===`
collapsible (hide)		`[%collapsible] ==== my text ====`
Keyboard, button, and menu macros		`:experimental: From menu: menu:File[Save] Hit kbd:[Ctrl-Z]`

AsciiDoc (.adoc files)
- Conversion
  - kramdown-asciidoc
    - Instal·lació
      - Dependències
        
        Mageia
        
        sudo dnf install ruby ...
      - gem install kramdown-asciidoc
    - Ús / Usage
      - kramdoc toto.md
        
        crearà: toto.adoc
  - asciidoc -> pdf
    - Install
      - sudo dnf install ruby-bigdecimal
      - gem install asciidoctor-pdf
    - Run
      - asciidoctor-pdf toto.adoc
- Extensions per a navegadors / Browser extensions
  - Asciidoctor Browser Extension
    - Mozilla: Asciidoctor.js Live Preview
- AsciiDoctor
  - Compare AsciiDoc to Markdown
    - AsciiDoc lists have extra space (when compared to Markdown output) #1087
    - [options="compact"]
      - not working on GitLab
  - Tables
    - Table Syntax and Attribute Reference
Markdown (wp)
- Mastering Markdown
- GitLab
  - GitLab Flavored Markdown (GLFM)
    - Diagrams and flowcharts
  - Handbook Markdown Guide (kramdown)
    - "This style guide is for about.gitlab.com. For styles in the GitLab UI, see the GitLab Flavored Markdown documentation."
    - Headings
      - "We don't use h1 headings, as they already are displayed on every page as its title, and we should not apply more than one h1 per page."
      - "Always start with h2 (##), and respect the order h2 → h3 → h4. Never skip the hierarchy level, such as h2 → h4."
    - Diagrams
    - Table of Contents (ToC)
      - [TOC]
      - [[_TOC_]]
  - ...
- Editors, viewers
  - StackEdit (editor and viewer)
  - Extensions for browsers
- Conversion to PDF
  - Extension for browser and print to pdf from browser menu
    - Markdown Viewer
      - per a veure taules amples:
        
        cliqueu sobre l'extensió (peça de trencaclosques a dalt a la dreta): «Obre l'extensió»
        
        seleccioneu Theme:
        
        GITHUB
        
        FULL
      - si les heu d'imprimir a pdf
        
        potser haureu de fer la mida del full més gran: A3
        
        imprimeix les imatges de fons
      - Problemes
        
        [TOC]
        
        [TOC] and json code sections not workign #101
    - GitLab Mardown Viewer
  - Pandoc
    - Installation
      - ...
    - Usage
      - pandoc -f markdown -t html5 toto.md --pdf-engine weasyprint -o toto.pdf
      - pandoc -f markdown -t html5 toto.md -o toto.pdf
        
        fa servir wkhtmltopdf
        
        Problemes
        
        Mageia 9: Bug 29326 - QtWebkit and dependencies (like wkhtmltopdf) should be dropped
        
        alternatives:
        
        weasyprint
        
        puppeteer
      - pandoc -f markdown toto.md --pdf-engine=xelatex -o toto.pdf
        
        fa servir LaTeX
        
        Problemes
        
        I can't find the format file `pdflatex.fmt'!
        
        Solució / Solution
        
        sudo dnf install texlive-dist
      - pandoc -f markdown toto.md --pdf-engine=xelatex -V mainfont="DejaVu Sans" -V fontsize="10pt" -o toto_latex.pdf
  - grip
    - render page using a local http server
    - Installation
      - pip install grip
    - Usage
      - grip your_markdown.md
  - markdown-pdf (nodejs)
    - Installation
      - npm install -g markdown-pdf
    - Usage
      - markdown-pdf /path/to/markdown
reStructuredText (wp)
- Llenguatges de marca / Mark languages
- used by Sphinx
- Quick reStructuredText
- Directives
  - .. <directive_name>:: :<option_1>: <option_1_value> :<option_2>: <option_2_value> <directive_content>
Kiwi (Python)

Desenvolupament / Development

Pàgines web dinàmiques / Dynamic web pages
Aplicacions web / Web applications
Rich Internet applications (RIA)
Apache Geronimo
- IBM Websphere

Ajax

Pàgines web dinàmiques / Dynamic web pages: Ajax
AJAX (wp)

XMLHttpRequest

CORS with Javascript / PHP

4.6 States: 0-4

readyState

0	UNSENT
1	OPENED
2	HEADERS_RECEIVED
3	LOADING
4	DONE

4.8 Response
- 4.8.1 The status attribute

ZK Simply Ajax and mobile
XMLHttp Requests for Ajax
APE (Ajax Push)

Google Code Labs
- Gears (geolocation)

Serveis web / Web services

Java Technology and Web Services
An Introduction to the Web Services Architecture and Its Specifications (Microsoft)
IDE
- Eclipse
- Mono FAQ: ASP.NET
- MS Visual
APIGee (for people who love APIs)

Message-oriented middleware

Info
- Message-oriented middleware (wp)
- Message broker (wp)
Protocols
- AMQP
- MQTT
- STOMP

Implementations

Apache ActiveMQ
- STOMP (wp)
- Cloud services
  - Amazon MQ
Apache Kafka (LinkedIn) (wp)
- Open-sourcing Kafka, LinkedIn’s distributed message queue (LinkedIn blog)
- Apache Kafka - distributed publish-subscribe messaging system - Morten Kjetland (Vimeo)
- Kafka: a Distributed Messaging System for Log Processing (pdf) (LinkedIn)
  - comparison with ActiveMQ and RabbitMQ
- RabbitMQ vs Kafka: which one for durable messaging with good query features?
- Clients
  - Clients (Apache wiki)
Apache Qpid (wp)

RabbitMQ

Cloud services
- Amazon MQ
Instal·lació / Installation
- Docker
  - rabbitmq
    - Implement RabbitMQ on Docker in 20 minutes
      - docker run --rm -it -p 15672:15672 -p 5672:5672 rabbitmq:3-management
      - http://localhost:15672/
        
        guest / guest
    - with management
      - docker run -d --hostname my-rabbit --name some-rabbit -p 5672:5672-p 15672:15672 rabbitmq:3-management
      - http://localhost:15672/
    - without management:
      - docker run -d --hostname my-rabbit --name some-rabbit -p 5672:5672 rabbitmq:3
    - logs from container:
      - docker logs some-rabbit
    - get the status, running rabbitmqctl from inside the container:
      - docker exec some-rabbit rabbitmqctl status
    - stop and remove container:
      - docker container stop some-rabbit
      - docker container rm some-rabbit
    - activate STOMP plugin (with web management):
      - mkdir rabbitmq-stomp
      - cd rabbitmq-stomp
      - create file Dockerfile with content:
        
        FROM rabbitmq:3.8-management RUN rabbitmq-plugins enable --offline rabbitmq_stomp
      - docker build -t rabbitmq-stomp .
      - docker run -d --name some-rabbit-stomp-p 15672:15672 -p 5672:5672 rabbitmq-stomp
      - http://localhost:15672/
- Mageia
  - Mageia >5, < 5
    - urpmi rabbitmq-server
  - Mageia ==5
    - Bug 15120 - rabbitmq-server new security issues fixed upstream in 3.4.1 and 3.4.3
    - Install from download:
      - rpm --import https://www.rabbitmq.com/rabbitmq-signing-key-public.asc
      - valid versions
        
        urpmi https://www.rabbitmq.com/releases/rabbitmq-server/v3.5.7/rabbitmq-server-3.5.7-1.noarch.rpm
      - invalid versions
        
        urpmi https://www.rabbitmq.com/releases/rabbitmq-server/v3.6.1/rabbitmq-server-3.6.1-1.noarch.rpm
        
        urpmihttps://www.rabbitmq.com/releases/rabbitmq-server/v3.6.0/rabbitmq-server-3.6.0-1.noarch.rpm
        
        WARNING: rabbitmq 3.6.0-1 requires erlang "R16B03","5.10.4"; Mageia 5 provides erlang R16B02
    - dependencies
      - urpmi erlang-mnesia erlang-os_mon erlang-xmerl erlang-inets erlang-eldap erlang-public_key erlang-ssl erlang-tools erlang-compilererlang-syntax_tools erlang-crypto
  - systemctl enable rabbitmq-server.service
  - systemctl start rabbitmq-server.service
  - systemctl status rabbitmq-server.service
- Ubuntu
  - sudo apt-get install rabbitmq-server
- CentOS
  - CentOS/Alma 8
    - Installing on RPM-based Linux (RedHat Enterprise Linux, CentOS, Fedora, openSUSE)
    - Passos / Steps
      1. curl -s https://packagecloud.io/install/repositories/rabbitmq/erlang/script.rpm.sh | bash
      2. sudo dnf install erlang
      3. curl -s https://packagecloud.io/install/repositories/rabbitmq/rabbitmq-server/script.rpm.sh | bash
      4. sudo dnf install rabbitmq-server
  - CentOS 7
    - Installing on RPM-based Linux (CentOS, Fedora, OpenSuse, RedHat)
    - Installing RabbitMQ on CentOS 7
    - sudo yum install rabbitmq-server
    - setsebool -P nis_enabled 1
    - systemctl enable rabbitmq-server.service
    - systemctl start rabbitmq-server.service
    - systemctl status rabbitmq-server.service
- AWS EC2
  - Amazon EC2 (official RabbitMQ site)
  - Problemes / Problems
    - database is stored in a dir named after the own ip address, and when the address changes (using AMI to creae new instances) the database is lost
      - RabbitMQ: users disappear / settings gone after reboot / restart
      - Solució / Solution
        
        echo NODENAME=rabbit@localhost >>/etc/rabbitmq/rabbitmq-env.conf
        
        systemctl restart rabbitmq-server.service
- Port
  - default: 5672
  - web management: 15672
  - What ports is RabbitMQ using?
- Accessible from outside (default is ipv6 ::1 only)
  - /etc/rabbitmq/rabbitmq.config (IMPORTANT: if this is the only entry in rabbit section, do not add comma at the end)
    - [ {rabbit, [ {tcp_listeners, [{"0.0.0.0", 5672}, {"::1", 5672}]} ...
  - from external computer:
    - nmap -p 5672 <rabbitmq_server_ip_address>
    - ...
- Check the status
  - systemctl status rabbitmq-server
    - should return:
      - Active: active (running)
    - and not:
      - Active: active (exited)
  - rabbitmqctl status
- Problemes / Problems
  - Troubleshooting Network Connectivity
  - resposta lenta / slow response
    - Solució / Solution
      - verify DNS resolution
        
        /etc/resolv.conf
        
        dig ...
  - timeout
  - check /var/log/rabbitmq/startup_{log, _err}
    - E.g.:
      - {"init terminating in do_boot",{could_not_start,rabbit,{{erlang_version_too_old,{found,"R16B02","5.10.3"},{required,"R16B03","5.10.4"}},{rabbit,start,[normal,[]]}}}}
  - rabbitmqctl ...
    - Error: unable to connect to node rabbit@localhost: nodedown
      - Solució / Solution
        
        rabbitmqctl -n rabbit@ip-172-31-20-115 -p celery_vhost list_queues
  - journalctl -u rabbitmq-server / systemctl status rabbitmq-server.service
    - Error description: noproc
      - Solució / Solution
        
        (?) Install erlang (e.g. sudo urpmi erlang)
    - Failed to start RabbitMQ broker
      - Solució / Solution
        
        setsebool -P nis_enabled 1
        
        moreover, if you tried to manually start rabbitmq server (sudo /usr/lib/rabbitmq/bin/rabbitmq-server), some files and dirs may have wrong owner (root instead of rabbitmq):
        
        journalctl -u rabbitmq-server
        
        {error,{could_not_write_file,"/var/lib/rabbitmq/mnesia/rabbit@ip-xxxx/cluster_nodes.config",...
        
        sudo rm -rf /var/lib/rabbitmq/mnesia/
        
        sudo systemctl start rabbitmq-server.service
    - Event crashed log handler: (empty /var/log/rabbitmq/rabbit@...log)
      - Solució / Solution
        
        update from erlang-...-R16B-03.16.el7.x86_64 -> erlang-...-R16B-03.18.el7.x86_64
Configuració / Setup
- timeout (Message acknowledgment)
  - l'haureu d'incrementar si teniu errors PreconditionFailed - PRECONDITION_FAILED amb Celery, quan s'executen tasques de llarga durada
  - per defecte és 1800000 ms (30 minuts)
  - per a incrementar-lo:
    - # Default RabbitMQ delivery acknowledgement timeout is 30 minutes consumer_timeout = 1800000
  - en un broker d'AmazonMQ: Edit a Amazon MQ for RabbitMQ Configuration Revision
- Cluster
  - Clustering Guide
Eines / CLI Tools
- rabbitmqctl
  - "Only root or rabbitmq should run rabbitmqctl"
  - rabbitmqctl status
  - rabbitmqctl list_queues name messages_ready messages_unacknowledged
  - rabbitmqctl list_exchanges
  - usuaris / users
    - rabbitmqctl report
    - rabbitmqctl list_users
    - rabbitmqctl add_user <username> <password>
    - rabbitmqctl add_vhost <my_vhost>
    - rabbitmqctl list_vhosts
    - rabbitmqctl set_permissions -p <my_vhost> <username> ".*" ".*" ".*"
    - set administrator privileges:
      - rabbitmqctl set_user_tags <username_administrator> administrator
  - queues
    - list
      - rabbitmqctl [-p my_vhost] list_queues
    - delete / purge (version >=? 3.5.4, otherwise, use rabbitmqadmin)
      - RabbitMQ - how do I delete all messages from a single queue
      - rabbitmqctl [-p my_vhost] purge_queue <queue_name>
- rabbitmqadmin
  - you need to activate rabbitmq_management plugin
  - download from github:
    - curl -O https://raw.githubusercontent.com/rabbitmq/rabbitmq-server/v3.12.x/deps/rabbitmq_management/bin/rabbitmqadmin
  - or download it from http://localhost:15672/cli/:
    - curl -O http://localhost:15672/cli/rabbitmqadmin
    - chmod +x rabbitmqadmin
  - used username has to have administration permission
  - queues
    - list queues
      - rabbitmqadmin -V celery_vhost -u <username_administrator> -p <my_password> list queues
      - rabbitmqadmin -V celery_vhost -u <username_administrator> -p <my_password> list queues <columns>
        
        rabbitmqadmin -V celery_vhost -u <username_administrator> -p <my_password> list queuesname messages_ready messages_unacknowledged
      - remote (e.g. AmazonMQ, where ssl is used):
        
        ./rabbitmqadmin --host=b-xxx.mq.eu-west-1.amazonaws.com --ssl --vhost=my_vhost --port=15671 --username=my_user --password=my_password list queues
        
        column "messages" will contain Total number of messages (Ready + Unacked), as seen in web admin tool
        
        ./rabbitmqadmin --host=b-xxx.mq.eu-west-1.amazonaws.com --ssl --vhost=my_vhost --port=15671 --username=my_user --password=my_password list queues name messages_ready messages_unacknowledged
        
        in celery:
        
        messages_ready: the number of tasks unassigned to a worker
        
        messages_unacknowledged: the number of tasks assigned to a worker, but not yet finished (see timeout, PRECONDITION_FAILED)
    - get last 10 messages
      - rabbitmqadmin -V celery_vhost -u <username_administrator> -p <my_password> get queue=<queue_name> count=10
    - purge a queue
      - rabbitmqadmin -V celery_vhost -u <username_administrator> -p <my_password> purge queue name=<queue_name>
Plugins
- STOMP
  - RabbitMQ STOMP Adapter
- rabbitmq_management (web / API interface)
  - /usr/lib/rabbitmq/bin/rabbitmq-plugins enable rabbitmq_management
  - >= v3
    - http://localhost:15672
      - guest / guest
      - or any other added user with administrator privileges
      - to see the queues, the user has to have permission for the specific virtual host
      - to delete a queue, select it and, at the bottom of the page: Delete / Purge
    - API clients
      - rabbitmqadmin
  - < v3
    - http://localhost:55672
Used by
- Celery

RabbitMQ Tutorials (rabbitmq-tutorials)

Concepts (AMQP 0-9-1)

tutorial		exchange		publish	bind	queue
		name	type	routing_key	routing_key
Hello World: The simplest thing that does something flowchart LR A((P)):::producerclass --> B[[Queue]]:::queueclass --> C((C)):::consumerclass classDef producerclass fill:blue,stroke:white,color:white; classDef queueclass fill:#f60,stroke:white,color:white; classDef consumerclass fill:green,stroke:white,color:white;	producer	''		'hello'		'hello'
	consumer					'hello'
Work queues: Distributing tasks among workers (the competing consumers pattern) flowchart LR A((P)):::producerclass --> B[[Queue]]:::queueclass B --> C((C₁)):::consumerclass B --> D((C₂)):::consumerclass classDef producerclass fill:blue,stroke:white,color:white; classDef queueclass fill:#f60,stroke:white,color:white; classDef consumerclass fill:green,stroke:white,color:white;	producer	''		'task_queue'		'task_queue'
	consumer					'task_queue'
Publish / Subscribe: Sending messages to many consumers at once flowchart LR A((P)) --> X{{X}} X --> B[[Q₁]] B --> C((C₁)) X --> E[[Q₂]] E --> D((C₂)) %% classdef classDef producerclass fill:blue,stroke:white,color:white; classDef queueclass fill:#f60,stroke:white,color:white; classDef consumerclass fill:green,stroke:white,color:white; classDef exchangeclass fill:red,stroke:white,color:white; %% style class A producerclass; class B,E queueclass; class C,D consumerclass; class X exchangeclass;	producer	'logs'	'fanout'
	consumer	'logs'	'fanout'		-	'amq.gen-xxxx'
Routing: Receiving messages selectively flowchart LR A((P)) --> X{{X}} X -- a --> B[[Q₁]] B --> C((C₁)) X -- a --> E[[Q₂]] X -- b --> E[[Q₂]] X -- c --> E[[Q₂]] E --> D((C₂)) %% classdef classDef producerclass fill:blue,stroke:white,color:white; classDef queueclass fill:#f60,stroke:white,color:white; classDef consumerclass fill:green,stroke:white,color:white; classDef exchangeclass fill:red,stroke:white,color:white; %% style class A producerclass; class B,E queueclass; class C,D consumerclass; class X exchangeclass;	producer	'direct_logs'	'direct'	severity
	consumer	'direct_logs'	'direct'		severity	'amq.gen-xxxx'
Topics: Receiving messages based on a pattern (topics) flowchart LR A((P)) --> X{{X}} X -- \.a. --> B[[Q₁]] B --> C((C₁)) X -- \..b --> E[[Q₂]] X -- c.# --> E[[Q₂]] E --> D((C₂)) %% classdef classDef producerclass fill:blue,stroke:white,color:white; classDef queueclass fill:#f60,stroke:white,color:white; classDef consumerclass fill:green,stroke:white,color:white; classDef exchangeclass fill:red,stroke:white,color:white; %% style class A producerclass; class B,E queueclass; class C,D consumerclass; class X exchangeclass;	producer	'topic_logs'	'topic'	x.y.z
	consumer	'topic_logs'	'topic'		x.y.z	'amq.gen-xxxx'
RPC: Request/reply pattern example flowchart LR A((Client)) -- request --> B[[RPC]] B --> C((Server)) C --> D[[Reply]] D -- reply --> A %% classdef classDef producerclass fill:blue,stroke:white,color:white; classDef queueclass fill:#f60,stroke:white,color:white; classDef consumerclass fill:green,stroke:white,color:white; classDef exchangeclass fill:red,stroke:white,color:white; %% style class A producerclass; class B,E queueclass; class C,D consumerclass; class X exchangeclass;	producer
	consumer

Python code example (using Pika)

	concepts	code
		producer	consumer
Hello World	channel queue exchange	#!/usr/bin/env python import pika connection = pika.BlockingConnection(pika.ConnectionParameters( host='localhost')) channel = connection.channel()	#!/usr/bin/env python import pika connection = pika.BlockingConnection(pika.ConnectionParameters( host='localhost')) channel = connection.channel()
		channel.queue_declare(queue='hello')	channel.queue_declare(queue='hello')
		channel.basic_publish(exchange='', routing_key='hello', body='Hello World!') print " [x] Sent 'Hello World!'"	print ' [*] Waiting for messages. To exit press CTRL+C' def callback(ch, method, properties, body): print " [x] Received %r" % (body,) channel.basic_consume(callback, queue='hello', no_ack=True) channel.start_consuming()
		connection.close()
Work queues	round-robin dispatching acknowledgements durability	#!/usr/bin/env python import pika import sys connection = pika.BlockingConnection(pika.ConnectionParameters( host='localhost')) channel = connection.channel()	#!/usr/bin/env python import pika import time connection = pika.BlockingConnection(pika.ConnectionParameters( host='localhost')) channel = connection.channel()
		channel.queue_declare(queue='task_queue', durable=True)	channel.queue_declare(queue='task_queue', durable=True)
		message = ' '.join(sys.argv[1:]) or "Hello World!" channel.basic_publish(exchange='', routing_key='task_queue', body=message, properties=pika.BasicProperties( delivery_mode = 2, # make message persistent )) print " [x] Sent %r" % (message,)	print ' [*] Waiting for messages. To exit press CTRL+C' def callback(ch, method, properties, body): print " [x] Received %r" % (body,) time.sleep( body.count('.') ) print " [x] Done" ch.basic_ack(delivery_tag = method.delivery_tag) channel.basic_qos(prefetch_count=1) channel.basic_consume(callback, queue='task_queue') channel.start_consuming()
		connection.close()
Publish / Subscribe	exchange direct topic header fanout temporary queues binding	#!/usr/bin/env python import pika import sys connection = pika.BlockingConnection(pika.ConnectionParameters( host='localhost')) channel = connection.channel()	#!/usr/bin/env python import pika connection = pika.BlockingConnection(pika.ConnectionParameters( host='localhost')) channel = connection.channel()
		channel.exchange_declare(exchange='logs', type='fanout')	channel.exchange_declare(exchange='logs', type='fanout')
			result = channel.queue_declare(exclusive=True) queue_name = result.method.queue
			channel.queue_bind(exchange='logs', queue=queue_name)
		message = ' '.join(sys.argv[1:]) or "info: Hello World!" channel.basic_publish(exchange='logs', routing_key='', body=message) print " [x] Sent %r" % (message,)	print ' [*] Waiting for logs. To exit press CTRL+C' def callback(ch, method, properties, body): print " [x] %r" % (body,) channel.basic_consume(callback, queue=queue_name, no_ack=True) channel.start_consuming()
		connection.close()
Routing	exchange direct	#!/usr/bin/env python import pika import sys connection = pika.BlockingConnection(pika.ConnectionParameters( host='localhost')) channel = connection.channel()	#!/usr/bin/env python import pika import sys connection = pika.BlockingConnection(pika.ConnectionParameters( host='localhost')) channel = connection.channel()
		channel.exchange_declare(exchange='direct_logs', type='direct')	channel.exchange_declare(exchange='direct_logs', type='direct')
			result = channel.queue_declare(exclusive=True) queue_name = result.method.queue
			severities = sys.argv[1:] if not severities: print >> sys.stderr, "Usage: %s [info] [warning] [error]" % \ (sys.argv[0],) sys.exit(1) for severity in severities: channel.queue_bind(exchange='direct_logs', queue=queue_name, routing_key=severity)
		severity = sys.argv[1] if len(sys.argv) > 1 else 'info' message = ' '.join(sys.argv[2:]) or 'Hello World!' channel.basic_publish(exchange='direct_logs', routing_key=severity, body=message) print " [x] Sent %r:%r" % (severity, message)	print ' [*] Waiting for logs. To exit press CTRL+C' def callback(ch, method, properties, body): print " [x] %r:%r" % (method.routing_key, body,) channel.basic_consume(callback, queue=queue_name, no_ack=True) channel.start_consuming()
		connection.close()
Topics	exchange topic	#!/usr/bin/env python import pika import sys connection = pika.BlockingConnection(pika.ConnectionParameters( host='localhost')) channel = connection.channel()	#!/usr/bin/env python import pika import sys connection = pika.BlockingConnection(pika.ConnectionParameters( host='localhost')) channel = connection.channel()
		channel.exchange_declare(exchange='topic_logs', type='topic')	channel.exchange_declare(exchange='topic_logs', type='topic')
			result = channel.queue_declare(exclusive=True) queue_name = result.method.queue
			binding_keys = sys.argv[1:] if not binding_keys: print >> sys.stderr, "Usage: %s [binding_key]..." % (sys.argv[0],) sys.exit(1) for binding_key in binding_keys: channel.queue_bind(exchange='topic_logs', queue=queue_name, routing_key=binding_key)
		routing_key = sys.argv[1] if len(sys.argv) > 1 else 'anonymous.info' message = ' '.join(sys.argv[2:]) or 'Hello World!' channel.basic_publish(exchange='topic_logs', routing_key=routing_key, body=message) print " [x] Sent %r:%r" % (routing_key, message)	print ' [*] Waiting for logs. To exit press CTRL+C' def callback(ch, method, properties, body): print " [x] %r:%r" % (method.routing_key, body,) channel.basic_consume(callback, queue=queue_name, no_ack=True) channel.start_consuming()
		connection.close()
RPC	callback queue correlation_id reply_to	#!/usr/bin/env python import pika import uuid class FibonacciRpcClient(object): def __init__(self): self.connection = pika.BlockingConnection(pika.ConnectionParameters( host='localhost')) self.channel = self.connection.channel() result = self.channel.queue_declare(exclusive=True) self.callback_queue = result.method.queue self.channel.basic_consume(self.on_response, no_ack=True, queue=self.callback_queue) def on_response(self, ch, method, props, body): if self.corr_id == props.correlation_id: self.response = body def call(self, n): self.response = None self.corr_id = str(uuid.uuid4()) self.channel.basic_publish(exchange='', routing_key='rpc_queue', properties=pika.BasicProperties( reply_to = self.callback_queue, correlation_id = self.corr_id, ), body=str(n)) while self.response is None: self.connection.process_data_events() return int(self.response) fibonacci_rpc = FibonacciRpcClient() print " [x] Requesting fib(30)" response = fibonacci_rpc.call(30) print " [.] Got %r" % (response,)	#!/usr/bin/env python import pika connection = pika.BlockingConnection(pika.ConnectionParameters( host='localhost')) channel = connection.channel() channel.queue_declare(queue='rpc_queue') def fib(n): if n == 0: return 0 elif n == 1: return 1 else: return fib(n-1) + fib(n-2) def on_request(ch, method, props, body): n = int(body) print " [.] fib(%s)" % (n,) response = fib(n) ch.basic_publish(exchange='', routing_key=props.reply_to, properties=pika.BasicProperties(correlation_id = \ props.correlation_id), body=str(response)) ch.basic_ack(delivery_tag = method.delivery_tag) channel.basic_qos(prefetch_count=1) channel.basic_consume(on_request, queue='rpc_queue') print " [x] Awaiting RPC requests" channel.start_consuming()

Mosquitto
- Info
  - Guía de introducción a MQTT con ESP8266 y Raspberry Pi
- Instal·lació / Installation
  - Mageia
    - urpmi mosquitto
  - Debian (Raspberry Pi)
    - ...
- Ús / Usage
  - servidor / server
    - systemctl enable mosquitto.service
    - systemctl start mosquitto.service
  - client
    - producer
      - mosquitto_pub -h <broker> -t <topic> -m <message>
      - rtl_433 -F "mqtt://localhost:1883"
    - consumer
      - mosquitto_sub -h <broker> -t <topic>
      - subscribe to all topics
        
        mosquitto_sub -v -t '#'
- ...
...

http://www.francescpinyol.cat/www.html
Darrera modificació: 14 de juliol de 2025 / Last update: 14th July 2025

Cap a casa / Back home.

Internet / WWW

Index

General

Aplicacions / Applications

CMS (content management)

Correu electrònic / E-mail

Dominis / Domains

Allotjament / Hosting

Feeds

Geolocalització / Geolocation

Grups de notícies / Newsgroups

Gestors de preferits / Bookmarks managers

Internet Services Providers (ISP)

Internet TV

Internet of things

Servidors FTP / FTP servers

Clients FTP / FTP Clients

Crawling

Robots

Servidors de correu / E-mail servers

Clients de correu electrònic / E-mail clients

HTML

CSS

XHTML

XML

JavaScript

DOM (Document Object Model)

Dart

Gràfics vectorials / Vectorial graphics

Adobe Flash

Altres llenguatges / Other languages (parsers)

Desenvolupament / Development

Serveis web / Web services

Message-oriented middleware