Internet / WWW
|
Index
|
|
|
General
|
|
|
|
|
|
|
|
|
Aplicacions / Applications
|
|
|
|
|
|
|
CMS (content management)
|
|
|
Correu electrònic / E-mail
|
|
|
Dominis / Domains
|
|
|
Allotjament / Hosting
|
|
alta domini
|
renovació (¤/any)
|
|
emmagatzematge
|
tràfic
|
correu
|
altres
|
preus
|
|
|
|
|
|
|
comptes
|
espai
|
MySQL |
PHP
|
One Click
|
mensual
|
anual
|
active24
|
8
|
16,99
8
|
Basic Hosting
|
5GB
|
il·limitat
|
|
|
x
|
x
|
x
|
4,99
1,99
|
|
swhosting
|
8
|
17,75
|
Petit L
|
2GB
|
5GB/mes
|
|
|
-
|
x
|
-
|
3,50
|
|
Don
Dominio (.cat)
|
9,95
|
22,95 17,95
|
Pla
Mini
|
100MB
|
5GB/mes |
5
|
50MB
|
-
|
-
|
-
|
0,08
|
1 |
Pla
Basic
|
1GB
|
12GB/mes
|
100
|
300MB
|
100MB
|
x
|
|
2,08
|
25
|
Dina
Hosting
|
8,95
|
18,60
|
Hosting
Personal
|
2GB
|
30GB/mes
|
20
|
|
|
|
|
4,50
|
|
Virtual
Pyme
|
8,95
|
18,95
|
Mínim
|
1GB |
5GB/mes |
10
|
|
|
|
|
2,45
|
|
CD
Mon
|
8
|
19,95
|
Pla
Start
|
1GB
|
10GB/mes
|
1
|
|
-
|
x
|
|
2,50
|
30
|
Entorno
Digital
|
8
|
19,99
|
Pla
Presència
|
2GB
|
10GB/mes |
5
|
|
-
|
-
|
|
3,80
|
|
|
Feeds
|
|
|
Geolocalització /
Geolocation
|
|
|
Grups de notícies / Newsgroups
|
|
|
Gestors de preferits /
Bookmarks managers
|
|
|
|
|
|
|
Internet Services Providers (ISP)
|
|
|
Internet TV
|
|
|
Internet of things
|
|
|
|
|
|
|
|
|
- TCP
and
UDP port numbers (Wikipedia)
- Internet Protocol
(IP)
- W3C
- HTTP
- HTTPS
- Securing
the web
- Servidors / Servers
- Clients
- Server Name Indicator (SNI) (wp)
- from nginx
docs:
- "With this configuration a browser receives
the default server’s certificate, i.e.
www.example.com regardless of the requested
server name. This is caused by SSL protocol
behaviour. The SSL connection is established
before the browser sends an HTTP request and
nginx does not know the name of the requested
server. Therefore, it may only offer the default
server’s certificate."
- "A more generic solution for running several
HTTPS servers on a single IP address is TLS
Server Name Indication extension (SNI, RFC
6066), which allows a browser to pass a
requested server name during the SSL handshake
and, therefore, the server will know which
certificate it should use for the connection."
- SNI in nginx
- HSTS
- Marca blanca / White label
- Let's
Encrypt
- wildcards
- Protocols
- ACME (Automatic Certificate
Management Environment)
- Documentation
- Challenge
types
-
ACME
challenge |
used by |
pros |
cons |
|
certbot
plugin |
|
|
| HTTP-01 |
- apache
- nginx
- webroot
- standalone
- manual
|
- It allows hosting providers to
issue certificates for domains
CNAMEd to them.
- ...
|
|
| DNS-01 |
|
|
|
|
|
|
|
- ACME Clients
- certbot
- Documentation
- Installation
- Compilation
- Requirements
- CentOS
sudo yum install
python2-mock
python-zope-interface python-zope-component
python-six python-setuptools
python2-acme
- Mageia
- Compile
certbot (recommended)
- or get it:
wget
https://dl.eff.org/certbot-autochmod a+x
certbot-auto./certbot-auto
-debug./certbot-auto
certonly
- Download
git clone
https://github.com/certbot/certbot.git
- Compile
cd certbot
sudo python setup.py install
- Compile
version >=0.22 (for wildcards)
cd certbotvirtualenv envsource env/bin/activatepip install --upgrade pip
pip install --upgrade
setuptools cd acmepython setup.py installcd ..python setup.py installsudo ./env/bin/certbot ...
- Configuració / Setup
- serveis
amb usuari no root / non-root services
- How
to use certs in non-root services?
- letsencrypt_change_permissions.sh
#!/bin/bash
# file permissions
chgrp ssl-cert -R
/etc/letsencrypt
chmod 2755 /etc/letsencrypt
chmod g=rX -R /etc/letsencrypt
groupadd --gid 3000 ssl-cert
change_letsencrypt_persmissions.sh
usermod -a -G ssl-cert nginx- crontab
certbot -n renew
--deploy-hook
/usr/local/bin/letsencrypt_change_permissions.sh
- Usage
|
info
|
usage
from
|
additional
setup for running server
|
|
|
CLI
options |
/etc/letsencrypt/renewal/your.domain.org.conf
|
Apache |
Nginx |
|
|
certbot
...
|
[renewalparams]
|
/etc/httpd/httpd.conf
|
/etc/nginx/nginx.conf
|
authentication
|
when
your html pages are on a static
server (hosted, AWS S3...)
|
--authenticator
manual
|
|
|
|
when
you have no running server, but
ports 80 and 443 are available. A
mini server will be started just for
the process.
E.g.: Postfix
email server
|
--authenticator
standalone |
authenticator
=
standalone
|
|
|
when
you have a running server (certbot
will generate some content locally,
in /path/to/.well-known/, which must
be available by http from
letsencrypt servers; this requires
additional setup of your server)
Can also be used to obtain a
certificate for Postfix
(-d mail.domain.org)
when an http server is already
running (even if it is running
with server_name
your.domain.org). But make
sure that you have a DNS entry: mail.domain.org
CNAME your.domain.org |
--authenticator
webroot
-w /path/to
|
authenticator
=
webroot
[[webroot_map]]
your.domain.org = /path/to
|
<VirtualHost *:80>
Alias
/.well-known /path/to/.well-known
<Directory
"/path/to/.well-known">
Options Indexes FollowSymLinks
MultiViews
AllowOverride All
Require all
granted
</Directory>
|
server
{
listen
80;
server_name
your.domain.org;
# letsencrypt
location
/.well-known {
alias /path/to/.well-known;
}
|
| when
you have a running Apache server |
--authenticator
apache |
authenticator
=
apache
|
|
|
| when
you have a running Nginx server |
--authenticator
nginx |
authenticator
=
nginx
|
|
|
|
|
|
|
|
|
installation
|
you
do not want to install the obtained
certificate
|
certonly
|
installer
=
None
|
|
|
you
want to install the obtained
certificate in the Apache config
|
--installer
apache
|
installer
=
apache
|
|
|
| you
want to install the obtained
certificate in the Nginx config |
--installer
nginx
|
installer
=
nginx
|
|
|
- Examples
- Just retrieve a certificate (a
nginx server is running):
webroot=/usr/share/nginx/html
letsencrypt_port=80
key_size=4096
email=me@my_company.com
domain=my_subdomain.my_company.com
sudo certbot certonly -n
--agree-tos --webroot -w
$webroot --http-01-port
$letsencrypt_port
--rsa-key-size $key_size
--email $email -d $domain
- just retrieve a wildcard
certificate (requires certbot
version >=0.22)
- install the required dns
plugin
sudo -i
letsencrypt_port=80
key_size=4096
email=me@my_company.com
domain='*.my_company.com'
certbot certonly -n
--agree-tos --dns-route53
--http-01-port
$letsencrypt_port
--rsa-key-size $key_size
--email $email -d $domain --server
https://acme-v02.api.letsencrypt.org/directory
- Problemes / Problems
The currently
selected ACME CA endpoint
does not support issuing
wildcard certificates.
- Solució / Solution
--server
https://acme-v02.api.letsencrypt.org/directory
- just retrieve a certificate using
AWS Route 53
- install the required dns
plugin
sudo -i
email=me@my_company.com
domain='mydomain.my_company.com'
certbot certonly
[--dry-run] -n --agree-tos
--dns-route53 --key-type ecdsa
--email $email -d
$domain
- Renewal
- Renewing
certificates
--pre-hook--post-hook--deploy-hook
- all certificates:
- specified certificates:
certbot certonly -n
--authenticator webroot
--webroot-path /path/to -d
your.domain.org
- Client
options
- Apache
- first time
certbot run --apache --http-01-port
80 --rsa-key-size 4096 --email your@email.org
-d your.domain.org- non interactive:
certbot run --apache -n
--agree-tos --http-01-port
80
--rsa-key-size 4096 --email your@email.org
-d your.domain.org
- first renewal (because apache authenticator is
not working for certain apache config)
certbot certonly -n --authenticator
webroot --webroot-path /path/to/ -d
your.domain.org
- next renewals
sudo certbot -n renew- Problemes / Problems
/etc/letsencrypt/options-ssl-apache.conf
not found
- Solució / Solution
sudo yum install
python2-certbot-apache ln -s
/usr/lib/python2.7/site-packages/certbot_apache/options-ssl-apache.conf
/etc/letsencrypt/options-ssl-apache.conf
Could not open configuration
file
/etc/letsencrypt/options-ssl-apache.conf:
Permission denied
- This happens because
/etc/letsencrypt is a symbolic link
to /mnt/nfs/letsencrypt, a
nfs-mounted point
- Solució / Solution
- Nginx (using webroot)
- first time
- certbot
certonly --webroot -w /path/to
--http-01-port 80 --rsa-key-size 4096
--email your@email.org -d your.domain.org
- non interactive:
- certbot
certonly -n
--agree-tos --webroot -w
/path/to --http-01-port
80--rsa-key-size 4096 --email
your@email.org -d your.domain.org
- renewal
- /etc/nginx/conf.d/my_site.conf
server {
# letsencrypt
location
/.well-known {
alias /path/to/.well-known;
}
...
}
- check that
/etc/letsencrypt/renewal/my.site.org.conf
contains:
...
[renewalparams]
authenticator = webroot
webroot_path = /path/to
installer = None
...
sudo certbot renew --post-hook
"systemctl restart nginx.service"- crontab
certbot -q
-n renew --post-hook
"systemctl restart nginx.service"
- Certificate installation in hosting
services
- Get only the certificate (follow the
instructions)
domain=my_subdomain.my_company.org
sudo certbot certonly --authenticator
manual -d $domain
- Install it:
- DonDominio
- Alojamiento web / Subdominios
- CERTIFICADO:
xclip -selection clipboard
<
/etc/letsencrypt/live/${domain}/cert.pem- CERTIFICADO:
CTRL-V
- CLAVE SSL:
xclip -selection clipboard
< /etc/letsencrypt/live/${domain}/privkey.pem- CLAVE SSL:
CTRL-V
- CERTIFICADOS DE AUTORIDAD:
xclip -selection clipboard
< /etc/letsencrypt/live/${domain}/chain.pem- CERTIFICADOS DE AUTORIDAD:
CTRL-V
- AWS
- if you want to use certificates for
LoadBalancer or CloudFront, consider using AWS ACM
instead
- EC2
- Elastic Load Balancer (ELB)
- Let's
encrypt
and ELBs?
- First time
- create a load balancer with a port 80
listener
- AWS IAM: grant permissions to
user/role that will be used in next
steps
elasticloadbalancing:CreateLoadBalancerListenersiam:GetServerCertificateiam:UploadServerCertificate- Example of policy:
{
"Version":
"2012-10-17",
"Statement": [
{
"Sid":
"",
"Effect":
"Allow",
"Action":
[
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:CreateLoadBalancerListeners",
"elasticloadbalancing:SetLoadBalancerListenerSSLCertificate"
],
"Resource":
[
"*"
]
},
{
"Sid":
"",
"Effect":
"Allow",
"Action":
[
"iam:ListServerCertificates",
"iam:GetServerCertificate",
"iam:UploadServerCertificate"
],
"Resource":
[
"*"
]
}
]
}
- from an instance (e.g. from an
instance behind the load balancer)
- get a LE certificate (IMPORTANT:
keys with 4096 bytes are not
allowed; only 2048 or
1024 (not allowed by Let's
Encrypt))
certbot ... --rsa-key-size
2048 ...
- upload the certificate to IAM (Upload
the
Certificate):
aws iam
upload-server-certificate
--server-certificate-name
${domain}.cert \
--certificate-body
file:///etc/letsencrypt/live/${domain}/cert.pem
\
--private-key file:///etc/letsencrypt/live/${domain}/privkey.pem
\
--certificate-chain file:///etc/letsencrypt/live/${domain}/chain.pem
- Get the ARN of the uploaded
certificate
ARN=$(aws --output json
iam get-server-certificate
--server-certificate-name
${domain}.cert | jq
'.ServerCertificate.ServerCertificateMetadata.Arn')
- Add
an
HTTPS Listener Using the AWS CLI
aws elb create-load-balancer-listeners
--load-balancer-name
my-load-balancer --listeners
Protocol=HTTPS,LoadBalancerPort=443,InstanceProtocol=HTTPS,InstancePort=443,SSLCertificateId=$ARN
- ...
- Renewal
- letsencrypt-aws
- Installation
git clone
https://github.com/alex/letsencrypt-aws.gitcd letsencrypt-awsvirtualenv envsource env/bin/activatepip install -r
requirements.txt deactivate
- First time
- ask ACME for a private key
source
env/bin/activate
python
letsencrypt-aws.py
register email@host.comdeactivate
- save it as
email_host_com.privkey.acme.pem
(locally or on AWS S3)
- create an IAM
policy (name it e.g.
LetsencryptAWS)
- Problemes / Problems
- The
requested
nginx plugin does not appear to be installed?
#1736
- Solution: manual request and installation
cd letsencrypt./letsencrypt-auto certonly -a
manual --rsa-key-size 4096 --email
your@email.org -d your.domain.org- /etc/nginx/nginx.conf (or
/etc/nginx/conf.d/your_site.conf)
ssl_certificate
/etc/letsencrypt/live/www.example.org/fullchain.pem;
ssl_certificate_key
/etc/letsencrypt/live/www.example.org/privkey.pem;
- Note: if you specify cert.pem instead of
fullchain.pem, browsers will work, but curl nor openssl s_client
won't:
- Content
negotiation
- JPEG JFIF
- SOAP (Simple
Object Access Protocol)
- Platform for Privacy
Preferences (P3P) (press
release)
- Resource Description Framework (wp)
(metadades/metadata)
- Media
Fragments URI (MFWG)
- WebVTT
(Web Video Text Tracks)
- Cross-Origin
Resource Sharing (CORS)
Access-Control-Allow-Origin:
http://foo.example/
- W3C TV
- IETF RFC
- Protocols de missatgeria /
Messaging protocols
-
- AMQP (Advanced Message Queuing Protocol)
(wp)
- MQTT (Message
Queue Telemetry Transport) (wp)
- Brokers
- Clients
- Android
- CLI
- mqtt
(Mosquitto)
mosquitto_pub (producer)mosquitto_sub (consumer)
- rtl_433
rtl_433 -F "mqtt://localhost:1883"rtl_433 -F
"mqtt://localhost:1883,retain=1"
- web
- MQTT.fx
(JavaFX)
- Instal·lació / Installation Download
- rpm
wget
http://www.jensd.de/apps/mqttfx/1.7.1/mqttfx-1.7.1-1.x86_64.rpm- Mageia
urpmi
mqttfx-1.7.1-1.x86_64.rpm
- Ús / Usage
- Python
- Other
- Projects
- STOMP
(Streaming Text Oriented Messaging Protocol) (wp)
- to connect to message-oriented
middleware
- STOMP
Protocol
Specification, Version 1.2
|
frame |
standard
headers
|
request headers |
response headers |
|
|
content-length |
content-type |
receipt |
|
|
| connection
frames |
CONNECT |
MAY |
|
|
accept-versionhostloginpasscodeheart-beat
|
|
CONNECTED |
MAY |
|
|
|
versionheart-beatsessionserver
|
| client
frames |
SEND |
MAY |
SHOULD |
MAY |
|
|
SUBSCRIBE |
MAY |
|
MAY |
destination idack: [auto, client,
client-individual]
|
|
UNSUBSCRIBE |
MAY |
|
MAY |
|
|
BEGIN |
MAY |
|
MAY |
|
|
COMMIT |
MAY |
|
MAY |
|
|
ABORT |
MAY |
|
MAY |
|
|
ACK |
MAY |
|
MAY |
|
|
NACK |
MAY |
|
MAY |
|
|
DISCONNECT |
MAY |
|
MAY |
|
|
| server
frames |
MESSAGE |
MAY |
SHOULD |
|
|
destinationmessage-idsubscriptionack
|
RECEIPT |
MAY |
|
|
|
|
ERROR |
MAY |
SHOULD |
|
|
|
- Server
- Client (producer: SEND; consumer: SUBSCRIBE)
- RadioDNS
- Notes
- frame
COMMAND
header1:value1
header2:value2
Body^@
telnet vis.muscradio.com 61613
Trying 81.20.48.151...
Connected to 81.20.48.151.
Escape character is '^]'.
CONNECT
^@ SUBSCRIBE
destination: /topic/fm/ce1/c479/09580/image
ack: auto
^@DISCONNECT
receipt:77
^@
- HTTP
headers
- HTTP
request
methods (wp)
-
method
|
description
|
note
|
usage
|
related HTML tag
|
|
|
|
safe
|
idempotent
|
|
HEAD
|
same as GET, but
without the body
|
|
|
|
|
GET
|
requests a
representation of the resource
|
only to retrieve
|
y
|
y
|
href
|
POST
|
submits data to be
processed
|
|
n
|
n
|
form method="post"
|
PUT
|
uploads a
representation of the resource
|
|
n
|
y
|
|
DELETE
|
deletes the resource
|
|
|
|
|
TRACE
|
echoes the received
request
|
|
|
|
|
OPTIONS
|
http methods
supported by the server
|
|
|
|
|
CONNECT
|
|
|
|
|
|
PATCH
|
apply partial
modifications to a resource |
|
|
|
|
- Ús / Usage
- curl -X
http_method
- Django
- POST
-
generation
|
received
by CGI
|
html form
|
curl
|
content-type
|
as standard input
|
<form
action="my_cgi.sh" method="post">
<input type="text" name="tata"
value="tata_value">
<input type="text" name="titi"
value="titi_value">
<input type="submit">
</form>
|
curl -i -X POST --data
'tata=tata_value' --data 'titi=titi_value'curl -i -H
"Content-Type:
application/x-www-form-urlencoded"
-d
'tata=tata_value&titi=titi_value' curl -i -X POST --data-binary
'tata=tata_value' --data-binary
'titi=titi_value'
|
application/x-www-form-urlencoded
|
tata=tata_value&titi=titi_value
|
<form
action="my_cgi.sh" method="post" enctype="multipart/form-data">
<input type="text" name="tata"
value="tata_value">
<input type="text" name="titi"
value="titi_value">
<input type="submit">
</form> |
curl -i -X POST -F
tata=tata_value -F titi=titi_value curl -i -X POST -H
'Content-Type: multipart/form-data'
-F
tata=tata_value -F titi=titi_value
|
multipart/form-data;
boundary=------------------------6e0b5ea0578a6399
|
--------------------------6e0b5ea0578a6399
Content-Disposition: form-data; name="tata"
tata_value
--------------------------6e0b5ea0578a6399
Content-Disposition: form-data; name="titi"
titi_value
--------------------------6e0b5ea0578a6399--
|
|
curl -i -X POST -H
'Content-Type: application/json'
--data
'{"toto":"toto_value","tata":"tata_value"}'url="https://127.0.0.1:8000/"
my_var="toto value";
json_code="{\"toto\":\"${my_var}\"}";
curl_options="-H
'Content-Type: application/json'
--data '$json_code'";
curl_command="curl -v -X POST
$curl_options '$url'";
eval "$curl_command"my_var="toto l'educat"; my_escaped_var=$(echo
"${my_var}" | sed
"s/'/'\\\''/g");
json_code="{\"toto\":\"${my_escaped_var}\"}";
curl_options="-H
'Content-Type: application/json'
--data '$json_code'";
curl_command="curl -v -X POST
$curl_options '$url'";
eval "$curl_command"
|
application/json
|
|
- HTTP status codes
- Response
Status Codes (RFC 7231, Section 6)
- Hyper Text
Coffee Pot Control Protocol (HTCPCP/1.0) (RFC 2324)
- List of HTTP status codes (wp)
- Setting
HTTP
Status Codes (Servlet tutorial)
-
family
|
description
|
code |
1xx
|
Informational
|
- 100 Continue
- 101 Switching Protocols
- 102 Processing
- 103 Early Hints
|
2xx
|
Success
|
- 200 OK
- 201 Created
- 202 Accepted
- ...
|
3xx
|
Redirection
|
- 300 Multiple Choices
- 301 Moved Permanently (POST
is switched to GET)
- ...
- 308 Permanent Redirect (do not switch methods)
|
4xx
|
Client error
|
- 400 Bad Request
- 401 Unauthorized
- 402 Payment Required
- 403 Forbidden
- 404 Not Found
- 405 Method Not Allowed
- ...
- 408 Request Timeout
- ...
- 409 Conflict
- ...
- 415 Unsupported Media Type
- ...
- 418 I'm a teapot
- ...
- 429 Too Many Requests
- ...
- 451 Unavailable For Legal Reasons
- ...
- 460
The load balancer received a request from a
client, but the client closed the connection
with the load balancer before the idle timeout
period elapsed.
|
5xx
|
Server error
|
- 500 Internal Server Error
- 501 Not Implemented
- 502 Bad Gateway
- 503 Service Unavailable
- 504 Gateway Timeout
- 505 HTTP Version Not Supported
- ...
|
- 4xx
- DOCTYPE
/ DTD
- Emmagatzematge / Storage
- Same
origin policy (wp)
- JSON-RPC
- Info
- Basics
- request frame:
method attribute
(mandatory)params attribute
(optional)
- if request frame has an optional
id
attribute, then a server must send a response
frame
- frames without
id are considered notification
frames; server does not response on notification frames
- communication is bidirectional
- Over
- Push
- Comet / Ajax push (wp)
- Categories
- Streaming
- Ajax with long polling
- Implementations
- WebSocket (wp)
(HTML5)
- RFC
6455
- WebSocket
in Django
- Websockets load-balancing
with HAProxy
- 9
Websocket Servers for Reliable Real-time Applications
- WebSocket
(Full Stack Python)
- Escalabilitat / Scalability
- Eines / Tools
- nginx
- ...
| format |
server |
client |
| raw |
CLI
Languages
|
CLI
wscat -c ws://127.0.0.1:3000websocat ws://127.0.0.1:3000iocat ws://127.0.0.1:3000- Weasel
Languages
|
| JSON-RPC |
|
Languages
|
| socket.io |
CLI
iocat --socketio -l
-p 3000
Languages
|
CLI
iocat --socketio
ws://127.0.0.1:3000
- as iocat is using
?EIO=3,
error "Unsupported protocol version"
is returned; to solve this, new (v4)
socket.io server has to be started
with option:
- js:
- const
io = new Server(server, {
allowEIO3:
true // false
by default
});
- python: ...
- iocat
--emit-key "chat message" --socketio
ws://127.0.0.1:3000/
Languages
|
-
|
info |
instal·lació / installation |
docs |
servidor / server |
client |
| Firefox |
|
|
|
|
Responses
|
| Weasel |
|
|
|
|
|
| websocat |
|
- Rust toolchain (Mageia)
curl --proto '=https'
--tlsv1.2 -sSf
https://sh.rustup.rs | shcargo install --features=ssl
websocat
- ...
|
|
|
websocat ws://127.0.0.1:1234/websocat
ws://ws.vi-server.org/mirror- websocat
--jsonrpc ...
|
| wscat |
|
npm install -g wscat |
|
|
wscat -c ws://websocket-echo.com
|
| ws |
|
|
|
|
|
| socket.io |
- overhead (not plain)
- features
|
|
|
- bash
- JavaScript (Node.js)
- socket.io
import
{ Server } from "socket.io";
const io = new Server(3000);
io.on("connection", (socket)
=> {
// send a
message to the client
socket.emit("hello", "world");
// receive
a message from the client
socket.on("howdy", (arg) =>
{
console.log(arg); // prints
"stranger"
});
});import
{ createServer } from "http";
import { Server } from
"socket.io";
const httpServer =
createServer();
const io = new
Server(httpServer, {
path:
"/my-custom-path/"
});const
express = require('express');
const app = express();
const http = require('http');
const server =
http.createServer(app);
const { Server } =
require("socket.io");
const io = new Server(server);
app.get('/', (req, res) =>
{
res.sendFile(__dirname
+ '/index.html');
});
io.on('connection', (socket)
=> {
console.log('a user
connected');
socket.on('disconnect', ()
=> {
console.log('user
disconnected');
});
socket.on('chat message',
(msg) => {
io.emit('chat message', msg);
});
});
server.listen(3000, () => {
console.log('listening
on *:3000');
});
- Python
- python-socketio
from
aiohttp import web
import socketio
sio = socketio.AsyncServer()
app = web.Application()
sio.attach(app)
async def index(request):
"""Serve
the client-side
application."""
with
open('index.html') as f:
return
web.Response(text=f.read(),
content_type='text/html')
@sio.event
def connect(sid, environ):
print("connect ", sid)
@sio.event
async def chat_message(sid,
data):
print("message ", data)
await
sio.emit("chat_message", data)
@sio.event
def disconnect(sid):
print('disconnect ', sid)
app.router.add_static('/static',
'static')
app.router.add_get('/', index)
if __name__ == '__main__':
web.run_app(app)
-
| deployment |
creation |
| aiohttp |
from
aiohttp import web
import socketio
sio =
socketio.AsyncServer()
app =
web.Application()
sio.attach(app)
async def index(request):
"""Serve the client-side
application."""
with
open('index.html') as f:
return
web.Response(text=f.read(),
content_type='text/html')
@sio.event
...
app.router.add_static('/static',
'static')
app.router.add_get('/',
index)
if __name__ == '__main__':
web.run_app(app)
|
| WSGI |
import socketio
sio =
socketio.Server()
app =
socketio.WSGIApp(sio)
|
| ASGI |
import socketio
sio =
socketio.AsyncServer(async_mode="asgi")
app =
socketio.ASGIApp(sio)
# not needed when
using Daphne
# uvicorn.run(app,
host='127.0.0.1',
port=5000)
|
|
|
- ...
- ...
|
- bash
- JavaScript (browser, Node.js, React
Native)
- socket.io-client
<!DOCTYPE
html>
<html>
<head>
</head>
<body>
<script
src="https://cdn.socket.io/4.5.4/socket.io.min.js"></script>
<script>
import {io} from
"socket.io-client";
const socket =
io("ws://localhost:3000")
// receive a message from
server
socket.on("hello", (arg) =>
{
console.log(arg); // prints
"world"
});
// send a message to the
server
socket.emit("howdy",
"stranger");
</script>
</body>
</html>import
{ io } from
"socket.io-client";
const socket =
io("https://example.com", {
path:
"/my-custom-path/"
});...
<body>
<ulid="messages"></ul>
<form
id="form" action="">
<input id="input"
autocomplete="off"
/><button>Send</button>
</form>
<script
src="https://cdn.socket.io/4.5.4/socket.io.min.js"></script>
<script>
var socket = io();
var messages =
document.getElementById('messages');
var form =
document.getElementById('form');
var input =
document.getElementById('input');
form.addEventListener('submit',
function(e) {
e.preventDefault();
if (input.value) {
socket.emit('chat message',
input.value);
input.value = '';
}
});
socket.on('chat message',
function(msg) {
var item =
document.createElement('li');
item.textContent = msg;
messages.appendChild(item);
window.scrollTo(0,
document.body.scrollHeight);
});
</script>
</body>
</html>
- Python
- python-socketio
import
asyncio
import socketio
sio = socketio.AsyncClient()
@sio.event
async def connect():
print("Received event:
connect")
print("my
sid is", sio.sid)
@sio.event
def connect_error(data):
print("The
connection failed!")
@sio.on('*')
async def catch_all(event,
data):
print(f"Received
event {event} with data:
{data}")
@sio.event
async def my_message(data):
print("Received event:
my_message, with data: ",
data)
await
sio.emit('my response',
{'response': 'my response'})
@sio.on("chat message")
async def on_missatge(data):
print("Received event: chat
message, with data: ", data)
@sio.event
async def disconnect():
print("Received event:
disconnect")
async def main():
await
sio.connect('http://localhost:3000')
await
sio.wait()
if __name__ == '__main__':
asyncio.run(main())
- ...
- ...
|
- Media
types
(ftp)
- Byte articles
- Cache
- vCard and vCalendar
- MIME-Types
- IANA
MIME Media Types
- HTTP
servers configuration
/etc/apache2/apache2.conf (or /etc/apache2/sites-available/my_config)
AddType
application/dash+xml .mpd .xml
- /etc/httpd/conf/mime.types
- IIS
- HTTP headers / File types
- Imatges / Images
|
|
|
- General
- Articles
- Programari / Software
- Server
- Client
- configuració / setup
- bash environment variable
- GUI
- Centre de control de Mageia
- ús / usage
- CLI
wget http://toto.org/page.htmlwget --proxy-user=USERNAME
--proxy-password=PASSWORD
http://toto.org/page.htmlcurl http://toto.org/page.htmlcurl --proxy-user user:password
http://toto.org/page.html
- aplicacions / applications
- Firefox
- Preferències -> Paràmetres de xarxa
- Chrome
- Configuració -> Sistema -> Obre la
configuració del servidor intermediari de
l'ordinador
- Discover
|
|
|
- Comparison
of
web servers (wp)
- Comparison
of
lightweight web servers (wp)
- Best
Practices
for Speeding Up Your Web Site (Steve Sounders)
- Servidors HTTPS / HTTPS
server
- Balanceig de càrrega / Load
balancing
- Django
deployment: scaling
- Solucions / Solutions
- DNS
- Maquinari / Hardware
- Programari / Software
- Cache on servers
- Servidors
de prova / Test servers
- Nginx (nginx.com) (wp)
- Compilació /
Compilation
- Init scripts
- show compilation options
- check config
- Instal·lació / Installation
- Mageia
urpmi nginxsytemctl enable nginxsytemctl start nginx
- Ubuntu
- CentOS
- How
To
Install Nginx on CentOS 7
- LEMP
server
on CentOS 7 with FastCGI
sudo yum install epel-releasesudo yum install nginx- Official
nginx repositories for CentOS
- /etc/yum.repos.d/nginx.repo
sudo systemctl start nginx.service- Firewall
(only needed if firewalld.service is running)
sudo firewall-cmd --permanent
--zone=public --add-service=http
sudo firewall-cmd --permanent --zone=public
--add-service=https
sudo firewall-cmd --reload
- SELinux
- SELinux
on HTTP servers
sudo setsebool -P
httpd_read_user_content 1- if only using nginx (not uwsgi)
- if using also uwsgi (unix socket permission
denied)
- SELinux
policy
for nginx and GitLab unix socket in Fedora
19
- disable Selinux:
- detect the problem
- access your website, to generate logs
sudo yum install
policycoreutils-python sudo grep nginx
/var/log/audit/audit.log | audit2allow
- solve the problem (do not use -M option,
as you will not be able to edit the te text
file to add rules)
sudo -i
grep nginx
/var/log/audit/audit.log | audit2allow
-m nginx > nginx.techeckmodule -M -m -o nginx.mod
nginx.tesemodule_package -o
nginx.pp -m nginx.modsemodule -i nginx.ppsetenforce 1
- example
- nginx.te
module
nginx 1.0;
require {
type httpd_t;
type init_t;
type nfs_t;
type user_home_t;
type var_lib_t;
type unlabeled_t;
class unix_stream_socket
connectto;
class file {read getattr open};
class lnk_file read;
class sock_file write;
}
#============= httpd_t
==============
allow httpd_t
init_t:unix_stream_socket
connectto;
allow httpd_t nfs_t:file {read
getattr open};
allow httpd_t nfs_t:lnk_file read;
allow httpd_t user_home_t:file
{read open};
allow httpd_t var_lib_t:sock_file
write;
allow httpd_t unlabeled_t:lnk_file
read;
allow httpd_t unlabeled_t:file
{read open};
- Problems
/ Problemes
- Django wsgi
- Documentation
(.org)
- Beginner's
Guide
- Modules
-
|
|
nginx.org
|
nginx.com |
|
|
|
3rd party
|
core
|
|
ngx_http_core_module
|
|
|
|
add_header (any header)
expires (cache headers)-
|
examples |
HTTP headers |
| add_header |
|
|
| expires |
# at now +
time (>=0)
expires 1d; |
Date: Thu, 23
Apr 2020 15:34:53 GMT
Last-Modified: Thu, 23 Apr
2020 15:19:47 GMT
Expires: Fri, 24 Apr
2020 15:34:53 GMT
Cache-Control: max-age=86400
|
# at now + time (<0)
expires -1d; |
Date: Thu, 23 Apr 2020
15:36:14 GMT
Last-Modified: Thu, 23 Apr
2020 15:19:47 GMT
Expires: Wed, 22 Apr 2020
15:36:14 GMT
Cache-Control: no-cache |
# at file modification +
time
expires modified 1d; |
Date: Thu, 23 Apr 2020
15:37:35 GMT
Last-Modified: Thu,
23 Apr 2020 15:19:47 GMT
Expires: Fri, 24 Apr
2020 15:19:47 GMT
Cache-Control: max-age=seconds_remaining_to_23_Apr_2020:15:19:47_GMT
|
# at precise date (local
time)
expires @15h30m; |
Date: Thu, 23 Apr 2020
15:39:34 GMT
Last-Modified: Thu, 23 Apr
2020 15:19:47 GMT
Expires: Fri, 24 Apr 2020 13:30:00
GMT
Cache-Control: max-age=seconds_remaining_to_24_Apr_2020:13:30:00_GMT
|
# at 1st Jan 1970
expires epoch; |
Expires: Thu,
01 Jan 1970 00:00:01 GMT
Cache-Control: no-cache
|
# at 31st Dec 2037
(max-age: 10 years)
expires max; |
Expires: Thu,
31 Dec 2037 23:55:55 GMT
Cache-Control:
max-age=315360000
|
# no cache headers
expires off; |
Expires
Cache-Control
|
|
|
|
|
ngx_http_rewrite_module
|
|
proxy
|
|
ngx_http_proxy_module
|
|
fastCGI
|
|
ngx_http_fastcgi_module
|
|
| authentication |
Access
|
ngx_http_access_module
|
|
Basic
|
ngx_http_auth_basic_module
|
|
JWT
|
ngx_http_auth_jwt_module
|
|
Digest
|
|
ngx_http_auth_digest
|
PAM
|
|
ngx_http_auth_pam_module
|
Request
|
ngx_http_auth_request_module
|
ngx_http_auth_request_module
|
languages
|
LUA
|
|
ngx_http_lua_module |
...
|
|
|
|
- Service setup
- NGINX
systemd service file
nginx.service
# https://www.nginx.com/resources/wiki/start/topics/examples/systemd/
[Unit]
Description=The NGINX HTTP and reverse proxy
server
After=syslog.target network.target
remote-fs.target nss-lookup.target
cloud-init.service
[Service]
Type=forking
PIDFile=/run/nginx.pid
TimeoutStartSec=200s
ExecStartPre=/usr/sbin/nginx -t
PrivateTmp=true
# other ExecStartPre scripts
ExecStartPre=/bin/bash
-c 'for script in /etc/nginx/execstartpre.d/*;
do $script; done'
ExecStart=/usr/sbin/nginx
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
[Install]
WantedBy=multi-user.target
- /tmp
- /usr/lib/systemd/system/nginx.service
- real /tmp dir for nginx is:
/tmp/systemd-private-...-nginx.service-.../tmp
- Configuració /
Configuration
- How
To Optimize Nginx Configuration
- Modificació
del fitxer de configuració / Modification of config file
- Modular
- /etc/nginx/
- nginx.conf
user nginx;
worker_processes 1;
http {
include
/etc/nginx/mime.types;
default_type
application/octet-stream;
# Load modular
configuration files from the
/etc/nginx/conf.d directory.
# See
http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
} # nginx-rtmp-module
rtmp {
# Load modular
configuration files from the
/etc/nginx/rtmp.conf.d directory.
include /etc/nginx/rtmp.conf.d/*.conf;
}
- execstartpre.d/
- script1_execstartpre.sh
- ...
- conf.d/
- http_server_1.conf
server {
listen
80;
server_name
localhost;
# Load modular
configuration files from the
/etc/nginx/conf.d/server_1.d/
directory.
include
/etc/nginx/conf.d/http_server_1.d/*.http_server.conf;
}
- http_server_1.d/
- dir_1_a.http_server.conf
location
/dir_1_a {
alias
/path/to/dir_1_a;
}
- rtmp_server_1.http_server.conf (if
using
# rtmp statistics
location /stat {
rtmp_stat all;
rtmp_stat_stylesheet stat.xsl;
}
location /stat.xsl {
root
/path/to/parent/dir/of/stat/xsl/file;
}
# rtmp control
location /control {
rtmp_control
all;
}
- https_server_2.conf
server {
# https
listen
443 ssl;
server_name
www.example.org;
ssl_certificate
/etc/letsencrypt/live/www.example.org/fullchain.pem;
ssl_certificate_key
/etc/letsencrypt/live/www.example.org/privkey.pem;
ssl_protocols
TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers
HIGH:!aNULL:!MD5;
...
}
- https_server_2.d/
- rtmp.conf.d
- rtmp_server_1.conf
server {
listen 1935;
ping 30s;
notify_method get;
# compatibility
with GStreamer:
publish_time_fix
off;
# Load modular
configuration files from the
/etc/nginx/rtmp.conf.d/wct-rtmp/
directory.
include
/etc/nginx/rtmp.conf.d/rtmp_server_1.d/*.rtmp_server.conf;
}
- rtmp_server_1.d/
- app_1_a.rtmp_server.conf
application
app_1_a {
live on;
wait_video on;
wait_key on;
}
- rtmp_server_2.conf
- rtmp_server_2.d/
- HTTPS
- Configuring
HTTPS
servers
- Client
Side
Certificate Auth in Nginx
- SNI (several domains with several
certificates)
- HSTS
- Protocols
- Ciphers
- nginx.conf
server {
# https
listen
443 ssl;
server_name
www.example.org;
ssl_certificate
/etc/letsencrypt/live/www.example.org/fullchain.pem;
ssl_certificate_key
/etc/letsencrypt/live/www.example.org/privkey.pem;
#ssl_protocols
TLSv1 TLSv1.1 TLSv1.2;
ssl_protocols
TLSv1.2;
ssl_ciphers
HIGH:!aNULL:!MD5;
# hsts
add_header
Strict-Transport-Security "max-age=31536000;
includeSubDomains; preload" always;
...
- Autenticació /
Authentication
- Debug
- A
debugging log (nginx.org)
- Debugging log for selected clients
- Logging to a cyclic memory buffer
- Debugging
NGINX (nginx.com)
- when building
./configure --with-debug ...- check it:
nginx -V 2>&1 | grep --
'--with-debug'
- nginx.conf
error_log
/path/to/log debug;error_log
/var/log/nginx/error.log debug;
- level possible values:
debug, info, notice,
warn, error, crit, alert, emerg
- tail -n 200 -f
/var/log/nginx/error.log
- Logs
- Logging
and
monitoring
- Error log
- Access log
- log_format
- default nginx.conf
http {
...
log_format
main '$remote_addr - $remote_user
[$time_local] "$request" '
'$status $body_bytes_sent "$http_referer"
'
'"$http_user_agent"
"$http_x_forwarded_for"';
...
}
- add request_time:
http {
...
log_format
main '$remote_addr - $remote_user
[$time_local] "$request" '
'$status $body_bytes_sent "$http_referer"
'
'"$http_user_agent"
"$http_x_forwarded_for" $request_time';
...
}
- access_log
- access.log columns when using awk
BEGIN
{FPAT="([^ ]+)|(\"[^\"]+\")"}; :
-
| $0 |
all line |
| $1 |
IP address |
| $2 |
- |
| $3 |
- |
| $4 |
datetime |
| $5 |
timezone |
| $6 |
request |
| $7 |
response code |
| $8 |
bytes |
| $9 |
referer |
| $10 |
user_agent |
$11
|
x_forwarded_for (real origin ip address)
|
$12
|
(request_time)
|
- count the most requested urls:
awk
'BEGIN {FPAT="([^ ]+)|(\"[^\"]+\")"}; {print $6}'
/var/log/nginx/access.log | sort | uniq -c | sort
-rn
- count the most requested urls inside
a specified time period:
awk 'BEGIN {FPAT="([^ ]+)|(\"[^\"]+\")"}; $4
~ /14\/Dec\/2021:13:5/ {print $6}'
/var/log/nginx/access.log | sort | uniq -c | sort
-n
- get requests with code 4xx and print
time, code, request:
awk 'BEGIN {FPAT="([^ ]+)|(\"[^\"]+\")"}; $4
~ /17\/Dec\/2021/ && $7 ~ /^4/ {print
$4,$7,$6}' /var/log/nginx/access.log
- Monitoratge / Monitoring
- ngxtop
- Instal·lació / Installation
- Ús / Usage
- only last entries (live)
- all log content
ngxtop --no-follow -i 'status >=
500' print request status http_referer
-
- your.conf
location /toto {
add_header x-peticio $request;
add_header x-temps $request_time;
add_header x-longitud $request_length;
return
200 'tot bé\n';
}
- Proxying to:
- Django
deployment
-
| nginx config |
backend
server
|
gateway
interface |
apps |
|
memcached |
|
|
|
|
proxy_pass ... |
ngx_http_proxy_module
|
|
|
|
|
location /mywebsocket {
proxy_read_timeout 1d;
proxy_send_timeout 1d;
proxy_pass
http://localhost:8123;
#proxy_pass
http://unix:/tmp/unixsocket_websocat;
proxy_http_version 1.1;
proxy_set_header Upgrade
$http_upgrade;
proxy_set_header Connection
"upgrade";
} |
|
websocket |
websocat
-s 8123- socket.io
node index.jspython server.py
sio =
socketio.Server()sio =
socketio.AsyncServer()
- ...
|
|
|
|
SCGI |
|
|
|
|
upload_pass ... |
upload-module
|
|
|
|
|
location ~ \.php$ {
include
/etc/nginx/fcgi_params; #or whatever you named
it
fastcgi_pass
127.0.0.1:9000;
} |
FastCGI
|
PHP
|
php-cgi -b 127.0.0.1:9000
|
CGI |
|
server {
...
location /myscript {
fastcgi_pass unix:/var/run/fcgiwrap.socket;
#
Fastcgi parameters, include the standard ones
include /etc/nginx/fastcgi_params;
fastcgi_param SCRIPT_FILENAME
/usr/local/bin/myscript.sh;
}
} |
|
FCGIWrap
|
systemctl start spawn-fcgi
|
|
|
# the upstream component nginx needs to
connect to
upstream django_mysite {
server
unix:///var/lib/uwsgi/mysite.sock;
}
# configuration of the server
server {
...
# Finally, send all
non-media requests to the Django server.
location / {
uwsgi_pass django_mysite;
include /etc/uwsgi/uwsgi_params
}
} |
UWSGI |
|
|
WSGI
|
- Django
- Socket.IO
WSGI
python wsgi_server.py
import
socketio
sio = socketio.Server()
app = socketio.WSGIApp(sio)
|
|
Gunicorn |
|
|
upstream channels-backend {
# use 127.0.0.1
instead of localhost to avoid errors related
to ipv6
server 127.0.0.1:9000;
}
server {
...
location / {
try_files $uri @proxy_to_app;
}
location @proxy_to_app {
proxy_pass http://channels-backend;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For
$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host
$server_name;
}
}
|
Daphne |
|
- daphne
-p 9000 myproject.asgi:application
- daphne
-b 0.0.0.0 -p 9000
myproject.asgi:application
- daphne
-e
ssl:443:privateKey=key.pem:certKey=crt.pem
my_project.asgi:application
- systemctl
start my_django_project.daphne.service
- [Unit]
Description=Daphne daemon (my_project)
After=network.target
[Service]
PIDFile=/run/daphne/pid
User=root
Group=root
WorkingDirectory=/path/to/my_project
ExecStart=/path/to/my_project/env/bin/daphne
--bind 0.0.0.0 --port 9000 --verbosity
0 my_project.asgi:application
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s TERM $MAINPID
Restart=on-abort
PrivateTmp=true
LimitNOFILE=32768
[Install]
WantedBy=multi-user.target
- daphne
-p 3000 asgi_server:app
- daphne
-b 0.0.0.0 -p 3000 asgi_server:app
- systemctl
start asgi_server.daphne.service
|
ASGI |
- Django
- my_project/my_project/asgi.py
import os
import django
from channels.routing import
get_default_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE",
"my_project.settings")
django.setup()
application =
get_default_application()
- Socket.IO
ASGI
python asgi_server.py
import
socketio
sio = socketio.AsyncServer()
app = socketio.ASGIApp(sio)
|
|
Uvicorn |
|
|
- FastCGI (CGI)
- fastcgi
module
(ngx_http_fastcgi_module)
- Understanding
and
Implementing FastCGI Proxying in Nginx
- FastCGI
example
- fastcgi_params
Versus
fastcgi.conf – Nginx Config History
- nginx
-
How to run a shell script on every request?
- FCGIWrap
(github)
- Documentation
- Installing
FcgiWrap
and Enabling Perl, Ruby and Bash Dynamic
Languages on Gentoo LEMP
- Installation
- Ubuntu
sudo apt-get install fcgiwrap- (no need to install spawn-fcgi, as
fcgiwrap already comes with a service)
sudo systemctl status
fcgiwrap.service
- Mageia
- from repo
- from source
- install dependencies
sudo dnf install
libfcgi-devel
- download and compile
git clone
http://github.com/gnosek/fcgiwrap.gitcd fcgiwrapautoreconf -i./configuremakesudo make install
- install and setup spawn-fcgi (needed?)
- CentOS
- from repo
sudo dnf install fcgiwrap
- from source
- install and setup spawn
fcgi
- install
- setup
- /etc/sysconfig/spawn-fcgi
#
You must set some working
options before the
"spawn-fcgi" service will
work.
# If SOCKET points to a
file, then this file is
cleaned up by the init
script.
#
# See spawn-fcgi(1) for
all possible options.
#
# Example :
#SOCKET=/var/run/php-fcgi.sock
#OPTIONS="-u apache -g
apache -s $SOCKET -S -M
0600 -C 32 -F 1 -P
/var/run/spawn-fcgi.pid --
/usr/bin/php-cgi"
FCGI_SOCKET=/var/run/fcgiwrap.socket
# if fcgiwrap was compiled
from source
#FCGI_PROGRAM=/usr/local/sbin/fcgiwrap
# if fcgiwrap was
installed from repo
FCGI_PROGRAM=
FCGI_USER=nginx
FCGI_GROUP=nginx
FCGI_EXTRA_OPTIONS="-M
0770"
# IMPORTANT: -F 1 means
only one worker!
OPTIONS="-u $FCGI_USER -g
$FCGI_GROUP -s
$FCGI_SOCKET -S
$FCGI_EXTRA_OPTIONS -F 1
-P /var/run/spawn-fcgi.pid
-- $FCGI_PROGRAM"- to redirect stderr
logs
to /var/log/nginx/error.log
(they will be shown as "
FastCGI
sent in stderr:")
- ...
OPTIONS="-u $FCGI_USER
-g $FCGI_GROUP -s
$FCGI_SOCKET -S
$FCGI_EXTRA_OPTIONS -F
1 -P
/var/run/spawn-fcgi.pid
-- $FCGI_PROGRAM -f"
- start
systemctl enable
spawn-fcgisystemctl start
spawn-fcgisystemctl status
spawn-fcgi- ...
- Problemes /
Problems
- Security issues
- Example to execute a script specified in the
url
- setup
- /etc/nginx/default.d/toto.conf
location
/cgi-bin/ {
# Disable
gzip (it makes scripts feel slower
since they have to complete
# before
getting gzipped)
gzip off;
# Set the
root to /usr/lib (inside this
location this means that we are
# giving
access to the files under
/usr/lib/cgi-bin)
root /usr/share/nginx;
# Fastcgi
socket
fastcgi_pass unix:/var/run/fcgiwrap.socket;
# Fastcgi
parameters:
include the standard ones
include /etc/nginx/fastcgi_params;
# Fastcgi
parameters:
adjust non standard
parameters (e.g. SCRIPT_FILENAME)
fastcgi_param
SCRIPT_FILENAME
$document_root$fastcgi_script_name;
# Fastcgi
parameters:
user-defined parameters
fastcgi_param
TOTO_PARAM $request_length;
}
sudo systemctl reload
nginx.service
- scripts
mkdir -p
/usr/share/nginx/cgi-bin- perl script
- /usr/share/nginx/cgi-bin/hello_world.cgi
#!/usr/bin/perl
-w
#
Tell perl to send a html
header.
# So
your browser gets the output
#
rather then
<stdout>(command line
# on
the server.)
print "Content-type:
text/html\n\n";
#
print your basic html tags.
# and
the content of them.
print
"<html><head><title>Hello
World!!
</title></head>\n";
print
"<body><h1>Hello
world</h1></body></html>\n";
chmod 755
/usr/share/nginx/cgi-bin/hello_world.cgi
- bash
script
- /usr/share/nginx/cgi-bin/toto.sh
#!/bin/bash
# as PATH is set in
/etc/init.d/functions, you may
need to complete it
# when you are calling other
scripts (e.g. in
/usr/local/bin) from here
PATH=${PATH}:/usr/local/bin
export $PATH
# call a script in
/usr/local/bin
my_script.sh
# content type must be
present, followed by an empty
line
# all lines must be ended
with \r\n (CR LF)
echo -e "\r"
# your text
echo "123"
echo "server software:
$SERVER_SOFTWARE"
echo "script filename:
$SCRIPT_FILENAME"
echo "toto param:
$TOTO_PARAM"
# print all available
environment variables
env
chmod 755
/usr/share/nginx/cgi-bin/toto.sh- Debug
- /usr/share/nginx/cgi-bin/toto_debug.sh
#!/bin/bash
-x
# adding -x will make bash
xtrace output (by default
written to stderr) to
appear in
/var/log/nginx/error.log,
# if option -f was
specified in spawn
config
echo -e "Content-type:
text/plain\n"
# log to stderr instead of
response body
(1>&2 echo
"[hello]")
...
- Parse
GET and POST parameters (using cgi_functions.sh)
- See also CGI
bash examples with POST
- /usr/share/nginx/cgi-bin/toto_with_params
#!/bin/bash
# if
cgi_functions.sh is
installed in
/usr/local/bin
PATH=/usr/local/bin:${PATH}
export PATH=$PATH
# register all GET and
POST variables
source cgi_functions.sh
cgi_getvars BOTH ALL
1>&2
# content type must be
present, followed by an
empty line
# all lines must be ended
with \r\n (CR LF)
echo -e "Content-type:
text/plain\r"
echo -e "\r"
# variables from QUERY and
POST are available as
shell variables
echo "var1: $var1"
echo "var2: $var2"
echo "var3: $var3"
- GET
curl -X GET
http://127.0.0.1/cgi-bin/toto_with_params?var1=value1
- application/x-www-form-urlencoded
curl -X POST
--data 'var3=value3'
http://127.0.0.1/cgi-bin/toto_with_params
- multipart/form-data
curl -X POST
-F var2=value2
http://127.0.0.1/cgi-bin/toto_with_params?var1=value1curl -X POST
-F var2=value2
http://127.0.0.1/cgi-bin/toto_with_params
- from local:
REQUEST_METHOD=POST
QUERY_STRING="var1=value1&var4=value4"
var2=value2 var3=value3
/usr/share/nginx/cgi-bin/toto_with_params- echo
"var2=value_a&var2=value_b&var3=value3"
| (REQUEST_METHOD=POST
CONTENT_TYPE=application/x-www-form-urlencoded
CONTENT_LENGTH=1000
./multi_cgi.sh)
- Allow only POST:
- /usr/share/nginx/cgi-bin/toto_only_post
#!/bin/bash
# register
all GET and POST variables
source cgi_functions.sh
cgi_getvars BOTH ALL
1>&2
if [[ $REQUEST_METHOD !=
"POST" ]]
then
echo
"Status: 405 Method Not
Allowed"
echo -e
"Content-type:
text/plain\r"
echo -e
"\r"
echo
"Method not allowed"
exit 0
fi # content
type must be present,
followed by an empty line
# all
lines must be ended
with \r\n (CR LF)
echo -e "Content-type:
text/plain\r"
echo -e "\r"
...
- Client
- Problemes / Problems
- SELinux
- 403 Forbidden
- 502 Bad gateway
- check that script returns as first
lines (optionally a
Status
line, containing return code number
and reason
phrase, see: ngx_http_fastcgi_hide_headers)
(note the empty line):
Content-type:
text/plain Status:
202 OK
Content-type: text/plain
- Example to execute a fixed script when
accessing a url:
- nginx
-
How to run a shell script on every
request?
- ...conf with standard parameters
location
/do_something/ {
# Disable
gzip (it makes scripts feel slower
since they have to complete
# before
getting gzipped)
gzip off;
# Set the
root to /usr/lib (inside this location
this means that we are
# giving
access to the files under
/usr/lib/cgi-bin)
root
/usr/share/nginx;
# Fastcgi
socket
fastcgi_pass unix:/var/run/fcgiwrap.socket;
# Fastcgi parameters,
include the standard ones
include /etc/nginx/fastcgi_params;
# Adjust non
standard parameters (SCRIPT_FILENAME)
fastcgi_param
SCRIPT_FILENAME
/path/to/fixed_script.sh;
#
user-defined parameters
fastcgi_param
TOTO_PARAM $request_length;
}
- fixed_script.sh (see also simple_cgi.sh
for an example with POST)
#!/bin/bash
# content type must be present,
followed by an empty line #
all lines must be ended with \r\n (CR LF)
echo -e "Content-type: text/plain\r"
echo -e "\r"
# execute some script
result=$(/path/to/do_something_script.sh)
echo "result:"
echo $result
# your text
echo "request method:
$REQUEST_METHOD"
echo "server software:
$SERVER_SOFTWARE"
echo "script filename:
$SCRIPT_FILENAME"
echo "toto param:
$TOTO_PARAM"
- fixed_script_with_specified http
response code
- conf with parameters from url
- location
/fast {
# Disable gzip (it
makes scripts feel slower since they
have to complete
# before getting
gzipped
gzip off;
# script will be
under
root
/usr/share/nginx;
# Fastcgi socket
fastcgi_pass
unix:/var/run/fcgiwrap.socket;
# Fastcgi
parameters, include the standard ones
include
/etc/nginx/fastcgi_params;
# script to be
executed
fastcgi_param
SCRIPT_FILENAME
/usr/share/nginx/cgi-bin/fixed_script.sh;
# parameters for
the script
fastcgi_param
REQUEST_URI $request_uri;
# parameters from
url
#
/fast/<param1>/<param2>/
if ($request_uri ~*
"/fast/([^/]*)/([^/]*)/$" ) {
set $param1 $1;
set $param2 $2;
}
fastcgi_param
PARAM1 $param1;
fastcgi_param
PARAM2 $param2;
}
- fixed_script with parameters
#!/bin/bash
# content type must be present,
followed by an empty line
# all lines must be ended with \r\n
(CR LF)
echo -e "Content-type: text/plain\r"
echo -e "\r"
echo "REQUEST_URI: ${REQUEST_URI}"
echo "PARAM1: ${PARAM1}"
echo "PARAM2: ${PARAM2}"
exit 0
- Example to get upload time:
- .../upload.conf
location /up {
# allow 100
Continue
client_max_body_size 20M;
# allow upload
sendfile on;
# Disable gzip (it
makes scripts feel slower since they
have to complete
# before getting
gzipped)
gzip off;
# Set the root to
/usr/lib (inside this location this
means that we are
# giving access to
the files under /usr/lib/cgi-bin)
root
/usr/share/nginx;
# Fastcgi socket
fastcgi_pass
unix:/var/run/fcgiwrap.socket;
# Fastcgi parameters,
include the standard ones
include /etc/nginx/fastcgi_params;
# Adjust non
standard parameters:
fastcgi_param
SCRIPT_FILENAME
/path/to/upload_time.sh;
# user-defined
parameters:
fastcgi_param
REQUEST_LENGTH $request_length;
fastcgi_param
REQUEST_TIME $request_time;
# user-defined
headers
add_header x-temps
$request_time;
add_header
x-longitud $request_length; }
- upload_time.sh
#!/bin/bash
# content type must be present,
followed by an empty line
# all lines must be ended with \r\n
(CR LF)
echo -e "\r"
# execute some script
result=$(/path/to/do_something_script.sh)
echo "result:"
echo $result
echo "request method:
$REQUEST_METHOD"
echo "request length:
$REQUEST_LENGTH"
echo "request time:
$REQUEST_TIME"
- Client
dd if=/dev/zero
of=megabyte.dat bs=1M
count=1 curl -i --data-binary
'@megabyte.dat'
http://<server>/up
- Cache
- CORS
- Upload
- Test upload speed
- Nginx
direct
file upload without passing them through backend
- nginx
upload
client_max_body_size issue
- client
- client_body_in_file_only
- files uploaded to
- ...conf
server {
listen 80;
server_name
localhost;
error_log
/var/log/nginx/error.debug.log debug;
#sendfile on;
location /up {
#auth_basic
"Restricted
Upload";
#auth_basic_user_file
basic.htpasswd;
#limit_except
POST
{ deny all; }
#client_body_temp_path
/tmp/;
#client_body_in_file_only on;
#client_body_buffer_size
128K;
# allow 100 Continue
client_max_body_size
20M;
#proxy_pass_request_headers on;
#proxy_set_header
X-FILE $request_body_file;
#proxy_set_body
off;
#proxy_redirect
off;
#proxy_pass
http://www.example.org/;
sendfile on;
add_header x-peticio $request;
add_header x-temps $request_time;
add_header x-longitud $request_length;
#root html;
#return 200 '$request_body_file';
return 200 'tot bé\n';
}
- Client (POST)
curl -i --data-binary '@myfile.png'
http://192.168.1.29/upload
- Problemes / Problems
- nginx-upload-module
- Upload
- Resumable upload
- nginx-upload-module
2.2 (github)
- Nginx
upload module (v 2.2.0)
- nginx.service
[Service]
...
# create hashed dirs for upload-module
(level1=1)
ExecStartPre=/bin/sh -c '/bin/mkdir -p
/var/www/uploads/{0..9}; chmod 777 -R
/var/www/uploads/{0..9}'
- Example configuration
# allow 100 Continue.
# if not specified
here, as general value, defaults to 1M,
and limits upload,
# even if
a greater vallue is specified inside a
location section
# (which only
controls return of 100 Continue message)
client_max_body_size
20M;
location /upload
{
# this value would response 100 for
files under 30M, but general
client_max_body_size 20M
# would not allow upload of files
between 20 and 30M:
# client_max_body_size
30M;
# Pass altered request body to this
location
upload_pass
@new_upload;
# Store files to this directory
# The directory is hashed:
# level1=1 (digit) : subdirectories 0 1
2 3 4 5 6 7 8 9 should exist
# (they can be created in nginx.service)
upload_store /var/www/uploads 1;
# Allow uploaded files to be read only
by user
upload_store_access user:r;
# Set specified fields in request body
# They are passed to backend as POST
multipart/form-data
upload_set_form_field
"${upload_field_name}_name"
"$upload_file_name";
upload_set_form_field
"${upload_field_name}_content_type"
"$upload_content_type";
upload_set_form_field
"${upload_field_name}_path
"$upload_tmp_path";
# Inform backend about hash and size of
a file
upload_aggregate_form_field
"${upload_field_name}_md5"
"$upload_file_md5";
upload_aggregate_form_field
"${upload_field_name}_size"
"$upload_file_size";
# pass fields from form
# pass all:
#upload_pass_form_field "(.*)";
# pass only submit and description:
upload_pass_form_field
"^submit$|^description$";
upload_cleanup 400 404 499 500-505;
}
# Pass altered
request body to a fastcgi
backend
location @new_upload {
# user-defined headers
add_header x-request-time $request_time;
add_header x-request-length
$request_length;
fastcgi_pass
unix:/var/run/fcgiwrap.socket;
# Fastcgi parameters, include the
standard ones
include /etc/nginx/fastcgi_params;
# Adjust non standard parameters
(SCRIPT_FILENAME)
fastcgi_param SCRIPT_FILENAME
/usr/local/bin/new_upload.sh;
# user-defined parameters:
fastcgi_param REQUEST_LENGTH
$request_length;
fastcgi_param REQUEST_TIME
$request_time;
}
- /usr/local/bin/new_upload.sh
#!/bin/bash
echo -e "Content-type: text/plain\n"
source cgi_functions.sh
# register all GET and POST variables
cgi_getvars BOTH ALL
echo "New upload
==============================="
export
- Other examples
- Compilation
-
|
client
|
response code
|
| POST
multipart/form-data |
curl -i
-X POST -H 'Content-Type:
multipart/form-data' -F toto='@myfile.png'
http://<server>/up |
200 OK
|
| POST
x-www-form-urlencoded |
curl -i
--data-binary '@myfile.png'
http://<server>/up |
415
Unsupported Media Type |
PUT
|
curl
--upload-file myfile.png http://<server>/up |
405 Not
Allowed |
- Resources
(.com)
- i18n
- 3rd party modules
- Problemes / Problemes
- check content of:
/var/log/nginx/error.log /usr/local/nginx/logs/error.log
- ...
- related to fcgiwrap
connect() to unix:/var/run/fcgiwrap.socket
failed (11: Resource temporarily
unavailable) while connecting to
upstream, client: ..., server: ..., request: ...,
upstream:
"fastcgi://unix:/var/run/fcgiwrap.socket:", host:
...
- uwsgi
with nginx (11: Resource temporarily
unavailable) while connecting to upstream)
- if, for some reason, a script called by
fastcgi get stuck, next requests are not being
served (netstats reports: CONNECTING), and they
are queued.
- This can be checked with:
netstat
--unix | grep fcgi | wc
- is there a limit?
- workaroud:
systemctl restart
spawn-fcgi.service
- to get the process that is blocking fastcgi
- debug:
netstat --unix ss --unixstracelsof | grep fcgi
- related to uWSGI (WSGI)
connect() to unix:///var/lib/uwsgi/....sock
failed (11: Resource temporarily
unavailable) while connecting to
upstream- connection to page gives "502 Bad Gateway"
systemctl status emperor.uwsgi.service
import memcache
ImportError: No module named memcache
- connection to page gives "502 Bad Gateway"
- /var/log/nginx/error.log
connect() to
unix:///etc/uwsgi/wct_backend.sock failed
(13: Permission
denied) while connecting to
upstream ...
- If
it
doesn't work
- Solució / Solution
- regenerate
nginx.pp file
sudo systemctl restart
emperor.uwsgi.servicesudo systemctl restart nginx.service
- connection to page gives "502 Bad Gateway"
- /var/log/nginx/error.log
connect() to
unix:///etc/uwsgi/wct_backend.sock failed
(2: No
such file or directory) while
connecting to upstream ...
- Solució / Solution
- check output from (maybe some
misspelling?):
sudo systemctl status
emperor.uwsgi.service
- check that uwsgi is installed in your
virtualenv
- check uwsgi
configuration
- related to Django and Daphne (ASGI)
connect() failed (111: Connection refused)
while connecting to upstream, client: ..., server:
..., request: ..., upstream: "http://[::1]:9000/ws/...",
host: ...
- IPv6 not activated
- Solució / Solution
- conf.d/my_nginx.conf
upstream
my-upstream {
# use 127.0.0.1
instead of localhost to avoid errors
related to ipv6
server
127.0.0.1:9000;
}
[alert]
2398#0: 1024 worker_connections are not enough
- Optimal
value for Nginx worker_connections
- Solució / Solution
- nginx.conf
events {
# this value must
not exceed:
# - max number of
open files for nginx user and
nginx.service:
# sudo su -
nginx -s /bin/bash -c "ulimit -Hn"
# grep
LimitNOFILE
/usr/lib/systemd/system/nginx.service
# - total range of
sockets to enable per IP: sysctl
net.ipv4.ip_local_port_range
# default value:
worker_connections 1024;
worker_connections 16384;
}
- set max number of open
files
- set total range of sockets to enable per
IP
sysctl
net.ipv4.ip_local_port_rangesysctl -w
net.ipv4.ip_local_port_range ...
[error] 22590#0: *344 recv() failed (104:
Connection reset by peer) while proxying upgraded
connection, client: ..., server: ..., request:
"GET ...", upstream: "http://127.0.0.1:9000/...",
host: ...
[info] 22590#0: *355 peer closed connection
in SSL handshake while SSL handshaking, client:
..., server: 0.0.0.0:443
accept4()
failed (24: Too many open files)
- Nginx:
24: Too Many Open Files Error And Solution
- nginx 24: Too many open
files
- see also: worker_connections are not enough
- Solució / Solution:
- increase limits for nginx user:
- present values:
sudo su - nginx -s /bin/bash -c
"ulimit -Hn"sudo su - nginx -s /bin/bash -c
"ulimit -Sn"
- set new values at os level for user nginx:
- /etc/security/limits.d/nginx.conf
# default:
nginx
hard
nofile 4096
nginx
hard
nofile 16384
# default:
nginx
soft
nofile 1024
nginx
soft
nofile 4096
- set new values at service level:
- /usr/lib/systemd/system/nginx.service
systemctl daemon-reloadsystemctl restart nginx.service
- check new values in running process:
nginx_pid=$(pgrep -f "nginx:
master process")cat /proc/${nginx_pid}/limits |
grep -E "Limit|Max open files"
Limit
Soft
Limit
Hard
Limit
Units
Max open
files
16384
16384
files
- perform stress tests
exec: fork failed (12: Cannot allocate memory),
client: ..., server: ...
nginx: [emerg] open() "/etc/nginx/nginx.conf"
failed (13: Permission denied)
- Solució / Solution
sudo setsebool -P
httpd_read_user_content 1
nginx: [emerg] bind() to 0.0.0.0:8000 failed
(13: Permission denied)
nginx: [emerg] bind() to 0.0.0.0:444 failed (13:
Permission denied)
- content is not cached
- Apache
HTTP Server (Apache
Software Foundation)
- Documentation
- How
to
configure Apache2 (ntu)
- Dynamic
Content with CGI (CGI)
- Environment
Variables in Apache
-
|
variable
|
bash
|
PHP
|
Python / Django
|
HttpRequest
|
CONTENT_LENGTHCONTENT_TYPEHTTP_ACCEPT_ENCODINGHTTP_ACCEPT_LANGUAGEHTTP_HOSTHTTP_REFERERHTTP_USER_AGENTQUERY_STRINGREMOTE_ADDRREMOTE_HOSTREMOTE_USERREQUEST_METHODSERVER_NAMESERVER_PORT
|
|
|
Django
request
and response objects
|
mod_ssl
|
SSL_CLIENT_S_DNSSL_CLIENT_S_DN- ...
|
|
|
|
CGI
(nginx fastcgi:
/etc/nginx/fastcgi_params)
|
fastcgi_param
QUERY_STRING
$query_string; fastcgi_param
REQUEST_METHOD
$request_method; fastcgi_param
CONTENT_TYPE
$content_type; fastcgi_param
CONTENT_LENGTH
$content_length; fastcgi_param
SCRIPT_NAME
$fastcgi_script_name; fastcgi_param
REQUEST_URI
$request_uri; fastcgi_param
DOCUMENT_URI
$document_uri; fastcgi_param
DOCUMENT_ROOT
$document_root; fastcgi_param
SERVER_PROTOCOL
$server_protocol; fastcgi_param
REQUEST_SCHEME
$scheme; fastcgi_param
HTTPS
$https
if_not_empty; fastcgi_param
GATEWAY_INTERFACE CGI/1.1; fastcgi_param
SERVER_SOFTWARE
nginx/$nginx_version; fastcgi_param
REMOTE_ADDR
$remote_addr; fastcgi_param
REMOTE_PORT
$remote_port; fastcgi_param
SERVER_ADDR
$server_addr; fastcgi_param
SERVER_PORT
$server_port; fastcgi_param
SERVER_NAME
$server_name;
|
|
|
|
user defined
(mod_env)
|
PassEnv ...
SetEnv
VARIABLE toto_valueUnsetEnv ...
|
|
|
|
- Apache
configuration
- MPM - Multi-Processing Modules (2.2)
(2.4)
- Sessions
- Rendiment / Performance
- Apache performance tuning (2.2)
(2.4)
- Càrrega del
servidor / Server load
- mod_status
- Debian / Ubuntu (Apache 2.4)
- /etc/apache2/mods-available/status.conf
<IfModule
mod_status.c>
# Allow server status reports
generated by mod_status,
# with the URL of
http://servername/server-status
# Uncomment and change the
"192.0.2.0/24" to allow access from
other hosts.
<Location /server-status>
SetHandler
server-status
#Require
local
#Require
ip 192.0.2.0/24
#
only from 188.79.*.* :
Require
ip 188.79
</Location>
# Keep track of extended status
information for each request
ExtendedStatus On
# Determine if mod_status displays the
first 63 characters of a request or
# the last 63, assuming the request
itself is greater than 63 chars.
# Default: Off
#SeeRequestTail On
<IfModule mod_proxy.c>
#
Show Proxy LoadBalancer status in
mod_status
ProxyStatus
On
</IfModule>
</IfModule>
sudo a2enmod status
- Old way
- httpd.conf
<IfModule
mod_status.c>
<Location
/server-status>
SetHandler server-status
Order
deny,allow
Deny
from all
allow
from 192.168.1
</Location>
ExtendedStatus On
</IfModule>
- get the status and refresh every 10 seconds:
http://your_ip/server-status?refresh=10
- MPM
parameters
- parsed
HTML (.shtml)
- TkApache
- Quick reference card
- User
Authentication
- Authentication
(Apache)
- mod_ssl
(https
config)
- Environment variables
SSL_CLIENT_S_DNSSL_SERVER_S_DN- ...
- Directives
- mod_auth_certificate
(*)
urpmi apache-mod_auth_certificate/usr/share/doc/apache-mod_auth_certificate/README/etc/httpd/conf/httpd.conf:
LoadModule
auth_certificate_module
/usr/lib/apache-extramodules/mod_auth_certificate.so
- Apache::Clean (mod_perl
Developer's
Cookbook)
- Introduction
to
Server Side Includes
- Rewrite
- Ten
Things You Didn't Know Apache (2.2) Could Do
- Apache Mobile
Filter (AMF)
- Modern
Mobile
Redirect Using .htaccess
RewriteCond %{REQUEST_URI} !^/mobile/.*$
RewriteCond %{HTTP_USER_AGENT}
"android|blackberry|ipad|iphone|ipod|iemobile|opera
mobile|palmos|webos|googlebot-mobile" [NC]
RewriteRule ^(.*)$ /mobile/ [L,R=302]
- Detect Mobile
Browser
- Fake http server with netcat
- Cherokee Web Server
(wp)
- Netscape Enterprise
Server
- NCSA HTTPd
- CERN httpd
- Servidors lleugers / Light servers
- Jetty Java HTTP Servlet
Server
- Web
Servers Comparison
- Creating
Net
Sites
- Designing Your
Web Site
- Databases
interfaces
- Utilitats / Utilities
- Gateway interface
- CGI (Common Gateway Interface)
- CGI on Apache
- FastCGI
- RFC
3875
"The Common Gateway Interface (CGI) Version 1.1"
- The
Web
Developer's Virtual Library course
- CGIHTML
(C routines)
- CGI
Developer's Guide
- Bash
- Bash.CGI
- cgi_functions.sh
#!/bin/bash
# https://oinkzwurgl.org/hacking/bash_cgi/
# Phillippe Kehi <phkehi@gmx.net>
and flipflip industries
# to test this function locally:
# source cgi_functions.sh
# QUERY_STRING="var1=value1" cgi_getvars
GET var1; echo $var1
#
QUERY_STRING="var1=value1&var2=value2"
cgi_getvars GET ALL; echo $var1; echo
$var2
#
CONTENT_TYPE="application/x-www-form-urlencoded"
... cgi_getvars POST ALL; echo $var1; echo
$var2
# to test this function locally:
# source cgi_functions.sh
# QUERY_STRING="var1=value1" cgi_getvars
GET var1; echo $var1
#
QUERY_STRING="var1=value1&var2=value2"
cgi_getvars GET ALL; echo $var1; echo
$var2
#
CONTENT_TYPE="application/x-www-form-urlencoded"
... cgi_getvars POST ALL; echo $var1; echo
$var2
# (internal) routine to store POST data
function cgi_get_POST_vars()
{
# check content type
# FIXME: not sure if we
could handle uploads with this..
if [ "${CONTENT_TYPE}"
!= "application/x-www-form-urlencoded" ]
&& ! [[ "${CONTENT_TYPE}" =~
^"multipart/form-data" ]]
then
# if, in /etc/sysconfig/spawn-fcgi, OPTIONS contains
-f,
# this echo can be seen at
/var/log/nginx/error.log
echo "[`basename $0`] WARNING: received
CONTENT_TYPE is ${CONTENT_TYPE}, but
implemented content types for POST are
only: "\
"application/x-www-form-urlencoded,
multipart/form-data" 1>&2
return
fi
# save POST variables
(only first time this is called)
[ -z
"$QUERY_STRING_POST" \
-a
"$REQUEST_METHOD" = "POST" -a ! -z
"$CONTENT_LENGTH" ] && \
read -N $CONTENT_LENGTH RECEIVED_POST
#echo "RECEIVED_POST:
$RECEIVED_POST"
if [[ "${CONTENT_TYPE}"
=~ ^"multipart/form-data" ]]
then
# export variables from multipart
boundary=$(echo $CONTENT_TYPE | awk -F=
'{print$2}');
#QUERY_STRING_POST=$(echo "$RECEIVED_POST"
| awk -v b=$boundary 'BEGIN
{RS=b"\r\n";FS="\r\n";ORS="&"} $1 ~
/^Content-Disposition/
{gsub(/Content-Disposition: form-data;
name=/,"",$1); gsub("\"","",$1); print
$1"="$3}')
# variables must be exported here, because
they can contain ampersands in value
eval $(echo "$RECEIVED_POST" | awk -v
b=$boundary 'BEGIN
{RS=b"\r\n";FS="\r\n";ORS=" "} $1 ~
/^Content-Disposition/
{gsub(/Content-Disposition: form-data;
name=/,"",$1); gsub("\"","",$1); print
"export "$1"=\""$3"\""}')
else
# take input string as is
QUERY_STRING_POST="$RECEIVED_POST"
fi
return
}
# (internal) routine to decode
urlencoded strings
function cgi_decodevar()
{
[ $# -ne 1 ] &&
return
local v t h
# replace all + with
whitespace and append %%
t="${1//+/ }%%"
while [ ${#t} -gt 0 -a
"${t}" != "%" ]; do
v="${v}${t%%\%*}" # digest up to the first
%
t="${t#*%}"
# remove digested part
# decode if there is anything to decode
and if not at end of string
if [ ${#t} -gt 0 -a "${t}" != "%" ]; then
h=${t:0:2}
# save first two chars
t="${t:2}"
# remove these
v="${v}"`echo
-e \\\\x${h}` # convert hex to special
char
fi
done
# return decoded string
echo "${v}"
return
}
# routine to get variables from http
requests
# usage: cgi_getvars method varname1 [..
varnameN]
# method is either GET or POST or BOTH
# the magic varible name ALL gets
everything
function cgi_getvars()
{
[ $# -lt 2 ] &&
return
local q p k v s
# get query
case $1 in
GET)
[ ! -z "${QUERY_STRING}" ] &&
q="${QUERY_STRING}&"
;;
POST)
cgi_get_POST_vars
[ ! -z "${QUERY_STRING_POST}" ] &&
q="${QUERY_STRING_POST}&"
;;
BOTH)
[ ! -z "${QUERY_STRING}" ] &&
q="${QUERY_STRING}&"
cgi_get_POST_vars
[ ! -z "${QUERY_STRING_POST}" ] &&
q="${q}${QUERY_STRING_POST}&"
;;
esac
shift
s=" $* "
# parse the query data
while [ ! -z "$q" ]; do
p="${q%%&*}" # get first part of
query string
k="${p%%=*}" # get the key (variable
name) from it
v="${p#*=}" # get the value
from it
q="${q#"$p"&*}" # strip first part
from query string
# decode and assign variable if requested
[ -n "$p" ] && [ "$1" = "ALL" -o
"${s/ $k /}" != "$s" ] && \
export
"$k"="`cgi_decodevar \"$v\"`"
done
return
}
- CGI
shell script using bash
- Bash
- How
to
parse $QUERY_STRING from a bash CGI script
- Creating
CGI Programs with Bash: Handling POST Data
curl -i -X POST --data 'tata=tata_value'
http://127.0.0.1/test
- Examples with
POST
- See also Parse
GET and POST parameters
- simple_cgi.sh
#!/bin/bash
code="200 OK"
echo -e "Status: ${code}\r"
echo -e "Content-type: text/plain\r"
echo -e '\r'
echo -e "REQUEST_METHOD:
${REQUEST_METHOD}\r"
echo -e "CONTENT_TYPE: ${CONTENT_TYPE}\r"
echo -e "CONTENT_LENGTH:
${CONTENT_LENGTH}\r"
echo -e "QUERY_STRING: ${QUERY_STRING}\r"
exit 0#!/bin/bash
code="200 OK"
echo -e "Status: ${code}\r"
echo -e "Content-type: text/plain\r"
echo -e 'Access-Control-Allow-Origin:*\r'
echo -e 'Access-Control-Allow-Credentials:
true\r'
echo -e 'Access-Control-Allow-Methods:
GET, POST, OPTIONS\r'
echo -e 'Access-Control-Allow-Headers:
DNT, X-CustomHeader, Keep-Alive,
User-Agent, X-Requested-With,
If-Modified-Since, Cache-Control,
Content-Type, Authorization \r'
echo -e '\r'
echo -e "REQUEST_METHOD:
${REQUEST_METHOD}\r"
echo -e "QUERY_STRING:
${QUERY_STRING}\r"
echo -e "CONTENT_TYPE: ${CONTENT_TYPE}\r"
echo -e "CONTENT_LENGTH:
${CONTENT_LENGTH}\r"
exit 0#!/bin/bash
# when called with:
# curl -i
"http://localhost/?a=b&c=d" --data
"toto=tu" --data "toto=ta"
# result is:
# REQUEST_METHOD: POST
# QUERY_STRING: a=b&c=d
# CONTENT_TYPE:
application/x-www-form-urlencoded
# CONTENT_LENGTH: 15
# CONTENT: toto=tu&toto=ta
code="200 OK"
echo -e "Status: ${code}\r"
echo -e "Content-type: text/plain\r"
echo -e "REQUEST_METHOD:
${REQUEST_METHOD}\r"
echo -e "QUERY_STRING: ${QUERY_STRING}\r"
echo -e "CONTENT_TYPE: ${CONTENT_TYPE}\r"
echo -e "CONTENT_LENGTH:
${CONTENT_LENGTH}\r"
read -N ${CONTENT_LENGTH} CONTENT
echo -e "CONTENT: ${CONTENT}\r"
exit 0 #!/bin/bash -x
echo -e "Content-type: text/plain\n"
echo "content type: ${CONTENT_TYPE}"
echo "QUERY_STRING_POST:"
cat
#!/bin/bash -x
echo -e "Content-type: text/plain\n"
echo "content type: ${CONTENT_TYPE}"
read -N $CONTENT_LENGTH QUERY_STRING_POST
echo "QUERY_STRING_POST:"
echo "$QUERY_STRING_POST"
- parse_multi.sh
#!/bin/bash
echo -e "Content-type: text/plain\n"
echo "CONTENT_TYPE: $CONTENT_TYPE"
echo "CONTENT_LENGTH: $CONTENT_LENGTH"
# get boundary from CONTENT_TYPE
boundary=$(echo $CONTENT_TYPE | awk -F=
'{print$2}')
echo "== boundary:"
echo "$boundary"
# read standard input
read -N $CONTENT_LENGTH QUERY_STRING_POST
echo "== received standard input:"
echo "$QUERY_STRING_POST"
# parse standard input in
multipart/form-data
post_params=$(echo "$QUERY_STRING_POST" |
awk -v b=$boundary 'BEGIN
{RS=b"\r\n";FS="\r\n";ORS="&"} $1 ~
/^Content-Disposition/
{gsub(/Content-Disposition: form-data;
name=/,"",$1); gsub("\"","",$1); print
$1"="$3}')
echo "== post_params:"
echo "$post_params"
exit 0
- Perl
- Python
- WSGI
(Python Web Server Gateway Interface)
- ASGI
(Asynchronous Server Gateway Interface)
- Servlets / JSP
servers
- C Server Pages
(CSP)
- MIME types configuration
|
Servidors FTP / FTP servers
|
- Clients FTP
- vsftpd
- ProFTPD
- Instal·lació / Installation
- CentOS
sudo yum install proftpdsystemctl start proftpd
- Mini-Howto
- Debugging
- /usr/lib/systemd/system/proftpd.service
ExecStart =
/usr/sbin/proftpd -d10
$PROFTPD_OPTIONS
- Configuring
a directory
- Virtual
users
- Passos / Steps
sudo -icd /etc; mkdir proftpd; cd proftpdwget
https://raw.githubusercontent.com/proftpd/proftpd/master/contrib/ftpasswdchmod +x ftpasswd ./ftpasswd --group -gid 9000 --name
ftp_users./ftpasswd --passwd --name user1 --home
/home --shell /bin/bash --uid 8000 --gid 9000/etc/proftpd.conf
# users
AuthUserFile /etc/proftpd/ftpd.passwd
AuthGroupFile /etc/proftpd/ftpd.group
systemctl restart proftpd.service
- Test
- AWS
and ProFTPD
- security groups
- proftpd.conf
# aws
PassivePorts
1024 1048
MasqueradeAddress
<your_public_ip_address>
- /etc/proftp.conf
DefaultRoot ~<Directory ~>
<Limit ALL>
DenyAll
</Limit>
<Limit DIRS READ>
AllowAll
</Limit>
</Directory>
<Directory ~/upload>
<Limit WRITE>
AllowAll
</Limit>
</Directory>
|
Clients FTP / FTP Clients
|
- sftp
- lftp
- download
lftpget ftp://user@remote_host/dir1/dir2/dir3/file1lftp ftp://user@remote_host/dir1 -e "get
dir2/dir3/file1; bye"- mirror
lftp ftp://user@remote_host/dir1 -e "mirror;
bye"
- upload
- reverse (-R) mirror
lftp -d -f commands.lftp 2>> toto.log
commands.lftp
open -u
user,password remote_host
set ftp:ssl-allow no
mirror -R
local_dir
- Exclude .git dir:
- lftp
exclude
syntax confusion
commands.lftp
open -u user,password
remote_host
set ftp:ssl-allow no
mirror
--exclude --exclude=^\.git/$ -R
local_dir remote_dir
- How
to
use rsync over ftp
- curlftpfs (mount locally)
- weex (updating web
pages)
- Python
|
|
|
- Pàgines
web
dinàmiques / Dynamic web pages
- PEAR packages
- PHP CLI (command line)
- PHP CLI
- Using
PHP from the command line
- Instal·lació / Installation
- Exemples / Examples: (-r to avoid
<?php...?>)
- php [-f] toto.php
php -r "mail('to_user@example.org','subjecte','cos
del
missatge','From:from_user@example.org' );"
- IDE
- PHP debug
php -i
/etc/php.ini
(Mageia: urpmi
php-ini) or /etc/php5/apache2/php.ini
(Debian/Ubuntu)
service httpd restart (or: sudo
service apache2 restart)- comprova la sintaxi / check syntax
urpmi php-cliphp -l toto.php
/etc/php.ini or /etc/php5/apache2/php.ini
- logs
- error_log
- Examples
error_log("You messed up!", 3,
"/var/tmp/my-errors.log"); error_log(print_r($variable,
TRUE)); error_log("You messed up!", 3,
"php://stdout");
- see also Apache
logs
- check the output
php -f toto.php- if result is empty and "
echo $?" returns
255, it could mean that dependencies not satisfied
- xdebug
- Instal·lació / Installation
- Mageia
- Ubuntu
sudo apt-get install php5-xdebug
- Eclipse
- Debugging
using XDebug
- Setup php ini file:
- Mageia:
/etc/php.d/A29_xdebug.ini
- Ubuntu:
/etc/php5/apache2/conf.d/20-xdebug.ini
xdebug.remote_enable =
1
xdebug.remote_handler = 'dbgp'
- An http server must be configured and running
- Eclipse: Window / Preferences / PHP / Servers
- Server
- Base URL: http://localhost
- Document Root: /var/www/html
- Debugger: XDebug
- Problems
- Progress tab: "waiting for xdebug session"
- Solution
xdebug.remote_handler = 'dbgp'
- kdevelop
- Debugging
remote
CLI with phpstorm
- Dependencies
- Errors
- Reference - What does this
error mean in PHP?
"Warning: cannot modify header information..."
- Si a php.ini hi ha:
error_reporting = E_ALL & ~E_DEPRECATED- i es produeix un "Notice:...", es posarà al text
de resposta, i el codi serà un 200, i ja no es
podrà canviar per un altre
- En canvi, si a php.ini hi ha:
error_reporting = E_ALL & ~E_DEPRECATED
& ~E_NOTICE- no s'escriurà cap "Notice:..." a la resposta, i el
codi HTTP es podrà canviar mitjançant un
"header("HTTP/1.0 204 No Response")"
- Penseu també a eliminar les línies en blanc o
retorns de carro fora de <php>
- i18n
- gettext
- PHP
internationalization
with gettext tutorial
- Localizing
PHP
Applications “The Right Way”, Part 1
- PHP
internationalization
with gettext tutorial
- WordPress
- Accept-Language (HTTP
request header)
- dependencies
- Mageia
urpmi php-gettext php-intl
- Steps
cd public- create a php file for testing:
- public/test_locale.php
<?php
// get language from browser preferences
(Accept-Language http header)
// only works for the first option in the
browser
$lang_from_http_header =
Locale::acceptFromHttp($_SERVER['HTTP_ACCEPT_LANGUAGE']);
echo "lang_from_http_header: " .
$lang_from_http_header . "<br/>";
putenv("LANGUAGE=".$lang_from_http_header);
// setup directory and domain (file name:
<domain>.mo)
$domain = "messages";
bindtextdomain($domain,"Locale");
bind_textdomain_codeset($domain, 'UTF-8');
textdomain($domain);
// why is this needed?
// this locale has nothing to do with the
language
// this locale must be installed in the
system
$locale = "en_US";
setlocale(LC_MESSAGES, $locale);
echo _("Hello world!");
//phpinfo();
?>
- extract all translatable strings to a portable
object template file:
xgettext --from-code=UTF-8 -o php.pot
*.phpfind . -iname "*.php" -exec xgettext
-a
-L PHP --from-code=UTF-8 -j -o php.pot {} \;
- create dir structure
mkdir -p Locale/ca/LC_MESSAGESmkdir -p Locale/fr/LC_MESSAGES...
- edit messages.pot
"Content-Type: text/plain; charset=UTF-8\n"
- create po files from pot
cp php.pot
Locale/ca/LC_MESSAGES/messages.po
- cd Locale/ca/LC_MESSAGES
- edit messages.po to update translations
- compile po to mo:
msgfmt -o messages.mo messages.po
- (?) restart Apache server
# systemctl restart httpd
- Problems
- PHP
internationalization
– i18n mechanisms tutorial
- How
to
use Locale::acceptFromHttp without a filter list?
- Smarty
- Smarty
- i18n
- smarty-gettext
- Smarty
gettext
plugin
- Installation
- cd <smarty>/plugins
- wget
https://raw.githubusercontent.com/smarty-gettext/smarty-gettext/master/block.t.php
- wget
https://raw.githubusercontent.com/smarty-gettext/smarty-gettext/master/function.locale.php
- Setup
cd public- get script to extract translatable strings
wget
https://raw.githubusercontent.com/smarty-gettext/smarty-gettext/master/tsmarty2c.phpchmod +x tsmarty2c.php
- edit your master tpl file:
{locale
path="Locale" domain="messages"}
- edit your_view.tpl
{t}Hello world
from Smarty!{/t}
- extract all translatable strings to a pot
file:
./tsmarty2c.php -o smarty.pot
<path_to>/views/
- merge pot files from *.php and *.tpl
xgettext --from-code=UTF-8 -o
php.pot *.phpmsgcat -o messages.pot php.pot
smarty.pot
- create po files from pot
cp messages.pot
Locale/ca/LC_MESSAGES/messages.po
cd Locale/ca/LC_MESSAGES- edit messages.po to update translations
- compile po to mo:
msgfmt -o messages.mo messages.po
- Next times:
- regenerate pot
xgettext --from-code=UTF-8 -o
php.pot *.php
./tsmarty2c.php -o smarty.pot
<path_to>/views/msgcat -o messages.pot php.pot
smarty.pot
- update po
msgmerge -U
Locale/ca/LC_MESSAGES/messages.po
messages.potmsgmerge -U
Locale/fr/LC_MESSAGES/messages.po
messages.pot...
- recompile mo
msgfmt -o Locale/ca/LC_MESSAGES/messages.mo
Locale/ca/LC_MESSAGES/messages.pomsgfmt -o Locale/fr/LC_MESSAGES/messages.mo
Locale/fr/LC_MESSAGES/messages.po
...
- Correu electrònic / E-mail
- curl-php
- PHP Curl with port number issue
- POST of array to djangorestframework,
in multipart (PhpMultiPartParser)
<?php
// http://stackoverflow.com/questions/3772096/posting-multidimensional-array-with-php-and-curl
function http_build_query_for_curl( $arrays, &$new
= array(), $prefix = null ) {
if ( is_object( $arrays ) ) {
$arrays =
get_object_vars( $arrays );
}
foreach ( $arrays AS $key =>
$value ) {
$k = isset(
$prefix ) ? $prefix . '[' . $key . ']' : $key;
if (
is_array( $value ) OR is_object( $value ) ) {
http_build_query_for_curl(
$value, $new, $k );
} else {
$new[$k]
= $value;
}
}
}
$ch = curl_init();
$url = "http://127.0.0.1:8000/mymodels/";
$myvalues = array("first", "second");
$post_fields = array("name" => "toto", "myvalues"
=> $myvalues );
http_build_query_for_curl( $post_fields, $post );
print_r($post);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HTTPHEADER,
array('Content-Type: multipart/form-data'));
curl_setopt($ch, CURLOPT_POST, sizeof($post_fields));
curl_setopt($ch, CURLOPT_POSTFIELDS, $post );
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$server_output = curl_exec($ch);
$info = curl_getinfo($ch);
if(curl_errno($ch)) {
echo "something was wrong in your
request";
}
curl_close($ch);
print $server_output;
?>
- Certificats
/
Certificates
- Cross-Origin
Resource
Sharing (CORS)
- Requests
with
credentials
- Server-side
Acess
Control (PHP examples)
- http://foo.example/ (Javascript)
var invocation = new XMLHttpRequest();
var url = 'https://bar.other/'
invocation.onreadystatechange = function() {
if (invocation.readyState!=4) return;
var is_ok = invocation.responseText=='OK';
if (is_ok) {
document.location.href = isok.html
} else {
document.location.href
= isnotok.html
}
}
invocation.open('GET', url, true ); invocation.withCredentials = "true";
invocation.send();
- https://bar.other/ (PHP)
<?php
header('Access-Control-Allow-Origin: http://foo.example/');
header('Access-Control-Allow-Credentials: true');
if ($_SERVER['SSL_CLIENT_VERIFY']=='SUCCESS') {
echo 'OK';
} else {
$sslcn = $_SERVER['SSL_CLIENT_S_DN_CN']; $msg = "$sslcn NOT_OK";
echo $msg;
}
?>
- Nota: si no es fan servir
Access-Control-Allow- (o bé
"Access-Control-Allow-Origin:
*"),
no estarà disponible responseText
a la URL origen. (Firefox: Cross-domain requests
with credentials return empty)
"Access-Control-Allow-Origin:
*" no pot estar lligat a withCredentials = "true"
- HTML
parsing
- PHP3 (es)
(ch)
- Databases interaction
- Emacs
modes
- Php Mode
(EmacsWiki)
cd /usr/share/emacs/24.2/lisp/progmodes/
wget
http://php-mode.svn.sourceforge.net/svnroot/php-mode/tags/php-mode-1.5.0/php-mode.el
- ~/.emacs
(autoload 'php-mode
"php-mode" "Major mode for editing php code." t)
(add-to-list 'auto-mode-alist '("\\.php$" .
php-mode))
(add-to-list 'auto-mode-alist '("\\.inc$" .
php-mode))
- LBD/EPFL
- Manual (local)
- Tutorial
- PHP Nuke
- Suhosin
(Hardened-PHP project - PHP security)
|
Crawling
|
|
|
|
|
- Search engine optimization (SEO) (wp)
|
Robots
|
|
|
|
|
- Criptografia
/
Cryptography
- Seguretat
en informàtica / Computer security
- Programari / Software
- HTTPS
- The World Wide Web
Security FAQ
- OpenSSL
- SSLeay
- Applications
with
SSL
- Check
- The Twenty Most
Critical Internet Security Vulnerabilities
- Autenticació i autorització d'usuari / User
authentication and authorisation
-
| authentication protocol |
uses authorisation
framework |
provider |
document served by provider |
client application |
mechanism |
| Kerberos |
|
|
|
|
|
| SAML |
|
SAML identity provider (IdP)
|
assertion (signed XML):
- subject information
- attributes
- issuer
- other info about authentication event
|
SAML service provider (SP) (always a web site)
|
the user (principal) is redirected from SP to IDP
for sign in |
| OpenID Connect (OIDC) |
OAuth 2.0 |
OpenID Provider (OP)
|
id_token (signed JSON)
- subject
- issuer
- authentication information
|
Relaying Party (RP) (a web or mobile application)
|
the user is redirected from RP to OP for sign in |
| Smart card |
|
|
|
|
|
|
|
|
|
|
|
- Info
- Standards
- Authentication protocols (l'usuari és qui diu que és)
- Authorisation frameworks
- Single sign-on (SSO)
- Info
- Autenticació social / Social authentication
- Info
- Clients
- Proveïdors
- Facebook
Login
- Access
tokens
- Graph
API (OAuth 2)
- Manually
Build
a Login Flow
- Creation and setup of a Facebook app:
- create an application at https://developers.facebook.com/
- configure the Site URL
- MyApps -> <your recently
created app>
- Settings -> Website ->
SiteURL
- for development:
- http://localhost:8000/
- this must match the url in
browser when trying to do
social login
- Note:
http://127.0.0.1:8000/ will
not work
- Examples
- Google
- Twitter
- Twitter
Developers
- Docs
- Basics
- Authentication
- OAuth
with the Twitter APIs
- Application-only
authentication: OAuth2
(bearer token)
- "Application-only
authentication is a
form of
authentication where
an application makes
API requests on its
own behalf, without
the user
context. This
method is for
developers that just
need read-only to
access public
information."
- endpoints:
POST
oauth2/tokenPOST
oauth2/invalidate_token
- Application-user
authentication: OAuth 1a
(access token for user
context)
- "The user
authentication
method of
authentication
allows an authorized
app to act on behalf
of the user, as the
user."
- endpoints:
POST
oauth/request_tokenGET
oauth/authorizePOST
oauth/access_token
- 4.1.
OAuth 1.0
- Media
- Developer utilities
- Sign
in with Twitter
- Create
an application
- Problemes / Problems
- Angular JS
- ...
- Mozilla
|
Servidors de correu / E-mail
servers
|
- Infraestructura /
Infrastructure
- DNS
-
Name
|
Type
|
Value
|
example.org.
|
MX
|
1
mail.example.org
|
example.org.
|
TXT
|
"v=spf1
include:... include:..." |
dddk._domainkey.example.org.
|
TXT
|
"v=DKIM1; k=rsa; p=..." |
- Ports:
- Seguretat / Security
- Seguretat / Security
- Info
- SSL / TLS (fixed ports: 465, 993, 995) -> use STARTTLS instead
- STARTTLS (wp: Opportunistic
TLS)
- passa de no segur a segur / upgrade plain text to
secure
- no calen ports específics / no need for specific
secure ports
- es pot fer servir amb: / can be used with: SMTP,
IMAP, POP3, ...
- DMARC
- SPF - Sender
Policy Framework
- DKIM -
DomainKeys Identified Mail
- Signatura digital de les capçaleres / Digital
signature of headers
- clau privada: ...
- clau pública: DNS TXT record
- About
DKIM (Google)
- DKIMCore
- DNS TXT record:
- Verificació / Verification
- Implementacions / Implementations
|
Clients de correu
electrònic / E-mail clients
|
- Obrir un adjunt winmail.dat
/ Open a winmail.dat attachment:
- Install tnef:
urpmi tnefapt-get install tnef
- Unpack attachement content:
- Command line
- How
do I test an imap server?
- Testing
POP3
and IMAP servers from the command line in CMD or bash
- imap:
- telnet mail.example.com 143
- ...
- telnet/nc to an SMTP server (after EHLO, responses
starting with 250 show available commands) (cannot
continue
a connection when STARTTLS has been invoked; use openssl
s_client or mailx
instead)
- port 25 (default)
- HELO (basic commands)
$ telnet
mail.example.org 25
Trying x.x.x.x...
Connected to mail.example.org.
Escape character is '^]'.
220 client.example.org ESMTP Postfix
HELO
mail.example.org
250 client.example.org
MAIL
FROM:<localuser@example.org>
250 2.1.0 Ok
RCPT
TO:<remoteuser@example.org>
250 2.1.5 Ok
DATA
354 End data with
<CR><LF>.<CR><LF>
To:<remoteuser@example.org>
From:<localuser@example.org>
Subject:First
test
Hi, you!
.
250 2.0.0 Ok: queued as 489C411EE7AC
QUIT
221 2.0.0 Bye
Connection closed by foreign host.
- EHLO (enhanced commands)
$ telnet
mail.example.org 25
Trying x.x.x.x...
Connected to mail.example.org.
Escape character is '^]'.
220 client.example.org ESMTP Postfix
EHLO
mail.example.org 250-client.example.org
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
...
- port 587 (must be activated; see Postfix
on port 587) and STARTTLS:
$ telnet mail.example.org
587 Trying x.x.x.x...
Connected to mail.example.org.
Escape character is '^]'.
220 client.example.org ESMTP Postfix
EHLO
mail.example.org 250-client.example.org
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
STARTTLS
...
- openssl s_client
openssl s_client -connect mail.example.org:587
-starttls smtpopenssl s_client -starttls smtp -connect
smtp.gmail.com:587 -crlf -ign_eof- Problemes / Problems
read:errno=0 (and connection is
closed):
- check, on postfix server:
/var/log/maillog
- mail / mailx
- Instal·lació / Installation
- Utilització / Usage
- Linux
and Unix mailx command
- mailx(1)
- Linux man page
- How
can
I use the “mail” command?
- enviament / sending (a local email server, e.g. Postfix,
must be running)
- enviament via un servidor smtp remot (p.ex. Postfix) /
sending though a remote smtp server (p.ex. Postfix)
- Sending
Email
from mailx Command in Linux Using Gmail’s SMTP
mail -s "subject" -S
from=my_user@toto.org -S
smtp=smtp://mail.toto.org
destination_user@example.org- STARTTLS and authentication
mailx -v -s "$EMAIL_SUBJECT"
-S smtp-use-starttls
-S ssl-verify=ignore
-S smtp-auth=login
-S smtp=smtp://smtp.gmail.com:587
-S
from="$FROM_EMAIL_ADDRESS($FRIENDLY_NAME)"
-S smtp-auth-user=$FROM_EMAIL_ADDRESS
-S
smtp-auth-password=$EMAIL_ACCOUNT_PASSWORD
-S ssl-verify=ignore
-S
nss-config-dir=~/.mozilla/firefox/yyyyyyyy.default/
$TO_EMAIL_ADDRESS
- recepció / reception
- MailX
Tutorial
mail
- next page:
z
- previous page:
z-
- delete:
d<from>-<to>
- list of messages (headers):
h
- ...
- Mozilla
Thunderbird (correu/mail)
- Command
line options
- create a new profile
thunderbird -CreateProfile my_profile- it will be created at:
~/.thunderbird/xxxxxxxx.my_profile/
- open gui profile manager
- open a profile
thunderbird -p my_profile
- Thunderbird 3
- Redacció d'un missatge en HTML quan per
omissió és en text pla / Compose a message in HTML when
default is plain text:
- Majúscula + click a "Redacta" / Shift + click on
"Compose"
- Expanded
columns
in folder pane
- Signatures
-
Thunderbird
- enllaços
cap
a firefox i altres aplicacions:
~/.thunderbird/.../prefs.js:
- Opening
hyperlinks
from some GTK-based applications doesn't work after
updating Firefox to a new version
-
Bug 58784 - Using
"Make firefox the default web browser" should use
/usr/bin/firefox not
/usr/lib(64)/firefox-<version>/firefox
$ gconf-editor
- /desktop/gnome/url-handlers/http/command
- enabled: no
- com que el valor per omissió és
"enabled=true", això crearà el fitxer
.gconf/desktop/gnome/url-handlers/http/%gconf.xml
(si no, no existeix el fitxer)
- llavors el thunderbird ens preguntarà amb quina
aplicació volem obrir els http: /usr/bin/firefox
- gconf:
~/.gconf/desktop/gnome/url-handlers/http/%gconf.xml
- Al Firefox: Edita / Preferències / Avançat / General /
Comprova-ho ara (predeterminat) (*)
- user_pref("network.protocol-handler.app.http",
"/usr/bin/xdg-open");
- user_pref("network.protocol-handler.app.http",
"/usr/bin/mozilla-firefox");
- user_pref("network.protocol-handler.app.https",
"/usr/bin/mozilla-firefox");
- user_pref("network.protocol-handler.app.ftp",
"/usr/bin/mozilla-firefox");
- user_pref("network.protocol-handler.app.smb",
"/usr/bin/konqueror");
- Locale-Switcher
Extension
- Tips
&
tricks
- Consells
i
trucs (Softcatalà)
- Use
different
Quote Level Colors
- ~/.thunderbird/xxx.default/chrome/userContent.css
/* Quote Levels Colors */
blockquote[type=cite] {
color: navy !important;
background-color: RGB(245,245,245) !important;
}
blockquote[type=cite] blockquote {
color: maroon !important;
background-color: RGB(235,235,235) !important;
}
blockquote[type=cite] blockquote blockquote {
color: green !important;
background-color: RGB(225,225,225) !important;
}
blockquote[type=cite] blockquote blockquote
blockquote {
color: purple !important;
background-color: RGB(215,215,215) !important;
}
blockquote[type=cite] blockquote blockquote
blockquote blockquote {
color: teal !important;
background-color: RGB(205,205,205) !important;
}
- Problem: "This body part will be downloaded on demand"
- View->Display Attachments Inline
- how
to
not include external images on html messages? - Mozilla
- moz-do-not-send true
- En recepció, caldrà acceptar "Mostra el contingut
remot"
- Lectura de fitxers mbox / Read
mbox files
- Reading
an mbox file with Thunderbird
- min_thunderbird.sh
#!/bin/bash
EXPECTED_ARGS=1
if (( $# != $EXPECTED_ARGS ))
then
cat <<EOF
Usage: `basename $0` profile_name
Create a thunderbird profile with given name, and
add a default mail setup, in order to read mbox
files.
EOF
exit 1
fi
# check that thunderbird is installed
if ! command -v thunderbird2 >/dev/null
2>&1
then
echo "ERROR: command
thunderbird does not exist. Please install it
(https://www.thunderbird.net/) before running this
script."
exit 1
fi
profile_name=$1
root_thunderbird_dir=${HOME}/.thunderbird
# create profile
echo "1. creating profile: ${profile_name}"
thunderbird -CreateProfile ${profile_name}
# get profile path
profile_dirname=$(find ${HOME}/.thunderbird -type
d -name "*\.${profile_name}")
echo " created: ${profile_dirname}"
# start thunderbird, to create dirs
echo "2. starting thunderbird to create dirs:
please just cancel and close"
thunderbird -p ${profile_name}
# backup orginal prefs.js
prefs_path=${profile_dirname}/prefs.js
original_prefs_path=${profile_dirname}/prefs.original.js
echo "3. backing up ${prefs_path} ->
${original_prefs_path}"
cp ${prefs_path} ${original_prefs_path}
# modify prefs.js
echo "4. modifying ${prefs_path} to add a minimal
mail config"
cat >>${prefs_path} <<EOF
user_pref("mail.root.none",
"${profile_dirname}/Mail");
user_pref("mail.accountmanager.accounts",
"account1");
user_pref("mail.accountmanager.localfoldersserver",
"server1");
user_pref("mail.account.account1.server",
"server1");
user_pref("mail.server.server1.directory",
"${profile_dirname}/Mail/Local Folders");
user_pref("mail.server.server1.directory-rel",
"[ProfD]Mail/Local Folders");
user_pref("mail.server.server1.hostname", "Local
Folders");
user_pref("mail.server.server1.name", "Carpetes
locals");
user_pref("mail.server.server1.nextFilterTime",
10);
user_pref("mail.server.server1.spamActionTargetAccount",
"mailbox://nobody@Local%20Folders");
user_pref("mail.server.server1.storeContractID",
"@mozilla.org/msgstore/berkeleystore;1");
user_pref("mail.server.server1.type", "none");
user_pref("mail.server.server1.userName",
"nobody");
EOF
# start thunderbird
echo "5. starting thunderbird: just check that
default mail folders have been created and close
it"
thunderbird -p ${profile_name}
# additional mbox files
echo "6. to read mbox files, put them on (or link
them from) ${profile_dirname}/Mail/Local Folders/
and restart thunderbird by using one of the
following:"
echo " thunderbird -p ${profile_name}"
echo " thunderbird -P"
echo "To delete profile ${profile_name}:"
echo " thunderbird -P"
echo "and select Delete Profile..."
echo "IMPORTANT: if you made a symbolic link to
your mbox file from ${profile_dirname}/Mail/Local
Folders/, you can choose Delete Files option.
Otherwise, choose Don't Delete Files."
exit 0
- Gmail
|
|
|
- List
of
web browsers (wp)
- Browserscope
- Browsers.com
- Herramientas
para una navegación satisfactoria (La Vanguardia)
- "A
virtually
secure browser" (pdf)
- Web
browser standards support
- Automatització / Automation
- User agents
- Plug-ins
- Configuració / Settings
-
Mozilla (*)
- Opera
- WaMCom (signText)
- El Navegador
- Netscape
- NCSA
Mosaic
- Konqueror
- Google
- Microsoft Internet Explorer
- Command line
- curl
- curl apis
- opcions / options:
-
group
|
option
|
description
|
input
|
-X
http_method |
method
|
-H
http_request_header |
request
header (e.g. to specify origin for CORS)
|
| output |
-i |
include the HTTP
response header in the output |
-I |
displays only
the HTTP
response
header in the output |
-s,
--silent |
silent |
-v |
verbose (show
also HTTP
request
headers) |
-O |
write output
to a local file named like the remote file
we get (like wget) |
|
-L |
follow new
location (3xx) |
|
-m,
--max-time <seconds> |
Maximum
time in seconds that you allow
the whole operation to take |
|
-w,
--write-out <format> |
Make curl display information on stdout
after a completed transfer. |
|
--referer
...
|
referer
|
authentication
|
-u
user:password |
|
| galetes
/ cookies |
-b filename /
<name>=<value>
|
read
from file / send a single cookie
|
-j
|
|
-c
filename
|
write to file
|
https
|
-k
|
trust
certificate
|
--cacert
ca_certificate
|
specify a
certificte for a trusted CA
|
|
...
|
|
- exemples / examples:
curl -X GET -H
'Accept: text/plain' http://example.com/curl -X GET -H
'Accept: application/json'
http://example.com/curl -X GET -u usuari:contrasenya
http://example.com/- POST data
curl -X POST
--data 'toto=true'
http://example.com/my_api/- JSON
curl -X
POST -H 'Content-Type: application/json'
--data-binary '{"toto":true}'
http://example.com/my_api/curl -X
POST -H 'Content-Type: application/json'
--data-binary '@toto.json'
http://example.com/my_api/my_value="toto"
my_json='{
"my_key":"'${my_value}'",
"my_other_key": "my_other_value"
}'
echo ${my_json} | curl ... -H
'Content-Type: application/json' -d
@-- nested json as multipart (e.g.: to
be able to upload an image at the same
time)
curl -X POST -H 'Content-Type:
multipart/form-data' -F name='nom' -F
location.name='Reus' -F
location.point='{"type":
"Point","coordinates":
[2.084205,41.473491]}}' -F
thumbnail=@cover.png http://example.com/my_api/
- GET JSON
curl -H 'Accept: application/json;
indent=4' -u admin:password
http://127.0.0.1:8000/users/curl -H 'Accept: application/json' -u
admin:password http://127.0.0.1:8000/users/ |
python -m json.toolcurl -X OPTIONS -H 'Accept:
application/json; indent=4' -u admin:password
http://127.0.0.1:8000/users/
curl -I -X GET http://example.com/- HEAD
- get modification time:
curl --head http://example.org/toto
2>/dev/null | awk
'/Last-Modified/ {$1="";print}'
- condition based on modification time:
- upload a file
curl --upload-file
my_file.png http://...
curl -F filedata=@localfile.jpeg
http://example.org/upload- specifying the filename that will be used to
store the file in server:
- percent
encoding
- How
to urlencode data for curl command?
- post params
# equivalent to https://... with
post params
curl ... --data-urlencode
"myparam=myvalue" ...
- query params
# equivalent to
https://.../?myparam=myvalue_urlencoded
curl ... --get
--data-urlencode "myparam=myvalue" ...
- Authentication
- Django
authentication
# login user1
key=$(curl -X POST -H 'Content-Type:
application/json' --data-binary
'{"username":"user1","password":"mypassword"}'
http://example.org/rest-auth/login/ | awk -F
':' '/key/{gsub(/[}"]/,"",$2); print $2}')
# get user details
curl -X GET -H 'Accept: application/json;
indent=4' -H "Authorization: Token $key" http://example.org/rest-auth/user/
- CORS
- In order to get a response header , you must
specify an Origin:
curl -i -H "Origin:
http://example.com" ...
- How
can
you debug a CORS request with cURL?
- regular CORS
curl -H "Origin:
http://example.com" --verbose ...
- preflight CORS
curl -H "Origin:
http://example.com" \
-H
"Access-Control-Request-Method: POST"
\
-H
"Access-Control-Request-Headers:
X-Requested-With" \
-X OPTIONS
--verbose ...
- referer
curl --referer
"http://example.org/index.html" ...
- download a file honouring Content-Disposition
- Galetes
/ Cookies
- HTTP
Cookies
- start cookie engine and read/write cookies to
a file
- write to file
- read from file
- send a single cookie
- If-Modified-Since
- check if the origin has a resource newer than
a specified date
curl -I -z "Fri, 16 Sep 2016
09:43:09 +0200" ...
curl -I -z
"$(date
-R -d 2016-09-16T07:43:09,328834376+0000)"
... curl -I -H "If-Modified-Since
$(date
-R -d 2016-09-16T07:43:09)"
...
- write-out
- print times and latencies
- How do I debug latency issues using curl?
curl -w "dns_resolution:
%{time_namelookup}, tcp_established:
%{time_connect}, ssl_handshake_done:
%{time_appconnect}, TTFB:
%{time_starttransfer}\n" ...
- reintents / retries
# retry:
# - every attempt will wait for up to 10
seconds (--max-time 10)
# - wait for 5 seconds before trying a
new attempt (--retry-delay 5)
# - up to 3 times (--retry 3)
# maximum total time: 10 + (first
retry:)5+10 + (second retry:)5+10 + (third
retry:)5+10 = 55s
retry_parms = "--connect-timeout 5
--max-time 10 --retry 3 --retry-delay 5
--silent --show-error"
- get http response code from bash
- How
to
split the HTTP error code from the contents in
cURL?
- How
to
evaluate http response codes from bash/shell
script?
--write-out
- Example:
# add http_code as
an extra final line, after the response
OUT=$(curl --silent --write-out
'\n%{http_code}' -X GET
http://www.example.org/)
# get the output code of the curl command
RET=$?
if [[ $RET -ne 0 ]]
then
echo "Error $RET"
else
# response is
everything but the last line
response=$(echo "$OUT"
| head -n-1)
echo "Response:
$response"
# http code is the last
line
http_code=$(echo "$OUT"
| tail -n1)
echo "http code:
$http_code"
if [[ $http_code ==
"200" ]]
then
# ok
...
else
# not ok
echo
"response: $response"
exit $http_code
fi
fi
- several retries (bash function)
#!/bin/bash
function send_curl {
# parameters:
# method
"url" "options"
# output is kept in
global variable curl_response
# return value:
# 0: ok (2xx)
# 1: reached maximum
number of retries
# other https codes
return the sum of digits. e.g.:
# 4: 400
# 7: 403
# 8: 404
# ...
# usage examples:
# send_curl GET
"http://192.168.1.100:8000/path/to/?toto=1&tata=2"
"-H 'Authorization: Token $key'"
# send_curl POST
"http://192.168.1.100:8000/path/to/" "-H
'Authorization: Token $key' -F
tete='titi'"
local method=$1
local url=$2
local options=$3
echo "method: $method"
echo "url: $url"
echo "options:
$options"
local
curl_command="curl --silent --write-out
'\n%{http_code}' -X $method $options
'$url'"
echo "curl_command:
$curl_command"
local return_code=1
local max_retries=10
local retries=0
curl_response=""
while (( retries <
max_retries ))
do
echo "[$retries] curl_commmand:
${curl_command}"
OUT=$(eval "$curl_command")
RET=$?
if [[ $RET -ne 0 ]]
then
echo "Error $RET"
else
http_code=$(echo
"$OUT" | tail -n1)
echo "http code: $http_code"
curl_response=$(echo
"$OUT" | head -n-1)
echo "curl_response: $curl_response"
# 2xx
if [[ $http_code =~ ^"2" ]]
then
echo
"ok"
return_code=0
else
#
return code is the sum of digits of
http_code
return_code=$(expr
$(echo $http_code | sed 's/[0-9]/ +
&/g' | sed 's/^ +//g'))
echo
"not ok: http_code=$http_code ->
return_code=$return_code"
fi
break
fi
sleep 2
(( retries++ ))
done
return $return_code
}
- json paginated result (e.g. from Django rest
framework result)
#!/bin/bash
method=$1
url=$2
options=$3
function get_page {
local method=$1
local url=$2
local options=$3
# execute curl request
curl_command="curl
--silent --write-out '\n%{http_code}' -X
$method $options '$url'"
echo "${curl_command}"
OUT=$(eval
"$curl_command")
RET=$?
# http response code
http_code=$(echo "$OUT"
| tail -n1)
# http response
curl_response=$(echo
"$OUT" | head -n-1)
# total number of
elements, including all pages
curl_count=$(echo
${curl_response} | jq '.count')
# link to next page
curl_next=$(echo
${curl_response} | jq -r 'if .links.next
then .links.next else empty end')
}
function get_all_pages {
local method=$1
local url=$2
local options=$3
# first request
get_page "${method}"
"${url}" "${options}"
retrieved_elements=$(echo ${curl_response}
| jq '.results')
cumulated_json="${retrieved_elements}"
# subsequent requests,
until no next is found
while [[ "${curl_next}"
]]
do
get_page "${method}" "${curl_next}"
"${options}"
retrieved_elements=$(echo ${curl_response}
| jq '.results')
# add new results to cumulated results
cumulated_json=$(echo ${cumulated_json} |
jq ". += ${retrieved_elements}")
done
echo "cumulated json:"
echo
"${cumulated_json}" | jq ''
}
get_all_pages "${method}" "${url}"
"${options}"
exit 0
- mime-type /
content_type
curl_get_command="curl
--silent --write-out '\n%{content_type}\n%{http_code}'
-X GET ..."
OUT=$(eval ${curl_get_command})
RET=$?
echo "OUT: $OUT"
if [[ $RET -ne 0 ]]
then
echo
" Error
$RET"
exit $RET
else
content_type=$(echo
"$OUT" | tail -n2 | head -1)
echo
"
content_type: $content_type"
http_code=$(echo "$OUT"
| tail -n1)
echo
" http_code:
$http_code"
response=$(echo "$OUT"
| head -n-1)
if [[ $http_code ==
"200" ]]
then
echo " ok"
else
echo "
response: $response"
fi
fi
- AWS
user data
# get user-data
# check if there is user_data
http_code=$(curl --write-out '%{http_code}\n'
--head -o /dev/null -s
http://169.254.169.254/latest/user-data/)
if [[ $http_code == "200" ]]
then
user_data=$(curl -s
http://169.254.169.254/latest/user-data/)
...
else
...
fi
- Django
REST
framework from curl
- Django
REST
framework test
- Django
Tastypie
from curl
- HTTPS
- cURL:
Adding/Installing/Trusting
New Self-Signed Certificate
- per a connectar-vos amb un servidor amb un
certificat de servidor emès per una CA / to
connect to a server with a server certificate
issued by a CA:
curl
... --cacert
ca_that_issued_the_server_certificate.crt
... https://...
- per a connectar-vos amb un servidor amb un
certificat autosignat
/ to connect to a server with a selfsigned
server certificate
- aconseguiu el certificat de servidor /
get the server certificate (
autosigned_server_certificate.crt)
curl
... --cacert
autosigned_server_certificate.crt ...
https://...
- o bé afegiu el certificat de la CA (o el
certificat autosignat del servidor) a la llista de
certificats de confiança / or add the CA
certificate (or the selfsigned server certificate)
to the list of trusted certificates:
- trust (selfsigned) certificates:
- Issues with Let's
Encrypt certificates
- GNU
Wget (at
GNU)
-
- galetes
/ cookies
- HTTPS
- client authentication:
wget --certificate=client_pem.crt
--private-key=client_pem.key
--ca-certificate=root_ca_pem.crt https://...
- recursiu / recursive:
-
- Clients FTP
- WGET
software for FTP and Web Auto-mirroring
(alternatives)
- Snarf
- Pavuk
- W3M
- Java-based
|
HTML
|
- Accessibilitat / Accessibility
- Characters, entities
- Desenvolupament /
Development
- General structure of a site
- HTML/CSS Templates
- Editors
- Mag's
Big
List of HTML Editors
- AdvaSoft AsWedit
- ASHE
- tkHTML
(tcl/tk)
- HTML
helper
mode for Emacs
- Quanta
- Blue
Griffon
- Installation
- Compilation
- Problemes /
Problems
- Kompozer
- Kompozer labs
- HTML
Timing
- Compilation
- Kompozer
Dev-Howto
- dependencies
- urpmi gcc-c++ lib64gnome-vfs2-devel
lib64IDL2-devel lib64xt-devel lib64ftgl-devel
lib64freetype2-devel lib64pangox-devel
lib64png12-devel
- svn checkout
https://kompozer.svn.sourceforge.net/svnroot/kompozer/trunk
kompozer
- svn patch this.patch
- kompozer/obj-kompozer/config/autoconf.mk
- XT_LIBS = -lX11 -lXt
FT2_LIBS = -lfreetype -lftgl
- cd kompozer
- cp mozilla/composer/config/mozconfig.fedora
mozilla/.mozconfig
- make -f client.mk build_all
- Problemes
/
Problems (vegeu / see: Bluegriffon)
- Bugs
- <object> makes title disappear from tab
- Nvu
-
- Atom
- gwrite
- Compilation
- dependencies
- urpmi python-distutils-extra intltool
- python-jswebkit
- urpmi python-cython lib64webkitgtk1.0-devel
lib64python-devel
- wget
https://gwrite.googlecode.com/files/python-jswebkit-0.0.3.tar.gz
- tar xvzf python-jswebkit-0.0.3.tar.gz
- cd python-jswebkit-0.0.3
- su; python setup.py install
- gwrite
- wget
https://gwrite.googlecode.com/files/gwrite-0.5.1.tar.gz
- tar xvzf gwrite-0.5.1.tar.gz
- cd gwrite-0.5.1
- su; python setup.py install; ./install
- Wix
(Flash)
- Aptana Studio
3 (Eclipse)
- prerequisites:
- Eclipse: Help -> Install new software -> Add
- Templates:
- Problems:
- Previously installed PyDev must be uninstalled (*)
- Javascript
- Manuals and specifications
- MathML (wp)
- MathML
(MDN - Mozilla)
- W3C Math Home
- Brief
Tutorial on MathML
- Specification
-
|
tokens |
entities |
Presentation MathML
<math mode="display"
xmlns="http://www.w3.org/1998/Math/MathML">
</math> |
<mi>x</mi> –
identifiers<mo>+</mo> –
operators<mn>2</mn> –
numbers<mtext>non zero</mtext>
– text<mrow> – a horizontal
row of items;<msup>, <munderover>,
and others – superscripts, limits over and
under operators like sums, etc.;<mfrac> – fractions;<msqrt> and
<mroot> – roots;<mfenced> - surrounding
content with fences, such as parentheses.
|
π→⁢...
|
| Content MathML |
|
|
- Lletres / Fonts
- Editors
- Conversion
from a Simple Syntax
-
|
|
|
|
| Client-side conversion |
TeXZilla (live
demo) (<x-tex>)
(based on LaTeX) (local)
|
<!DOCTYPE html>
<html lang="en">
<head>
<meta
charset="utf-8">
<title>Example of
x-tex</title>
<link
rel="stylesheet" type="text/css"
href="css/x-tag-components.css"/>
<link
rel="stylesheet" type="text/css"
href="css/tex.css"/>
<script
type="text/javascript" src="js/x-tag-components.js"></script>
<script
type="text/javascript"
src="js/tex.js"></script>
</head>
<body>
<h1>Example of
x-tex</h1>
<p>Inline
example:
<x-tex>\frac{x^2}{a^2} +
\frac{y^2}{b^2} = 1</x-tex>
</p>
<p>Block example:
<x-tex
display="block">\frac{x^2}{a^2} +
\frac{y^2}{b^2} = 1</x-tex>
</p>
</body>
</html> |
Inline example: \frac{x^2}{a^2}
+ \frac{y^2}{b^2} = 1
Block example:\frac{x^2}{a^2}
+ \frac{y^2}{b^2} = 1
|
|
AsciiMath |
|
|
|
LaTeXMathML |
|
|
| Server-side conversion |
|
|
|
- Firemath - The
Equation Editor (Mozilla Firefox add-on)
- Navegadors / Browsers
- TeXZilla en local
- Per a poder tenir tex.{css,js},
x-tag-components.{css,js} en local:
cd ~/src/git clone
https://github.com/fred-wang/x-tex.gitcd /path/to/html/root/css/
ln -s
~/src/x-tex/demo/x-tag-components.css .ln -s ~/src/x-tex/src/tex.css .
cd /path/to/html/root/js/
ln -s
~/src/x-tex/demo/x-tag-components.js .ln -s ~/src/x-tex/src/tex.js
- Perquè tots els navegadors puguin presentar el codi
escrit en MathML, cal afegir un pollyfill:
-
| project |
script |
usage |
|
<script
src="https://fred-wang.github.io/mathml-warning.js/mpadded-min.js"></script> |
deixa triar a l'usuari |
| mathml.css |
<script
src="https://fred-wang.github.io/mathml.css/mspace.js"></script> |
|
| MathJax |
<script
src="https://fred-wang.github.io/mathjax.js/mpadded-min.js"></script> |
|
- Firefox
- Mathzilla:
Collection of MathML-related add-ons
- HTML5
(draft)(WhatWG
Web
applications 1.0) (wp)
- Estil / Style
- Metadades incrustades / Embedded metadata
- Testers / Validations
- Utilitats / Utilities
|
CSS
|
- Web Style Sheets home page
- CSS (All standards
and drafts) (W3C)
-
- CSS usage in Confluence Export PDF:
- CSS validator
- CSSED - "A GTK-2 CSS
Editor"
- CSS (wp)
- Mozilla
CSS
support chart
-
selector
|
usage
|
example
|
usage from HTML
|
|
for all elements of
specified type
|
h1 {...}
|
<h1>
|
id selector
|
for a single, unique
element
|
#toto {...}
|
<...
id="toto">
|
class
selector
|
for any type of element,
with this class
|
.toto {...} |
<p
class="toto">
<h1 class="toto">
...
|
only for the specified
type, with this class
|
h1.toto {...} |
<h1
class="toto"> |
- Inserció / Insertion
- external
- toto.css
h1
{
/* comment */
color:red;
text-align:center;
}
- html
<head>
<link rel="stylesheet" type="text/css"
href="toto.css" />
</head>
- internal
<head>
<style>
h1
{
color:red;
text-align:center;
}
</style>
</head>
- inline
<body>
<h1 style="color:red;
text-align:center;"/>
</body>
- Posicionament
/
Positioning
-
|
description
|
original space
preserved
(does it affect the position of other elements?)
(is on the normal flow?)
|
move with scroll
|
static
|
default
|
yes
|
yes
|
fixed
|
relative to browser
window
|
no
|
no
|
relative
|
relative to its
normal position
|
yes
|
yes
|
absolute
|
relative to the first
parent element that has a position other than
static.
If no such element is found, the containing block is
<html>
|
no
|
yes
|
- Margins
- Exemples / Examples
- HbbTV
- mostra el contorn de l'element / Outline:
outline: #00dd00 dotted
medium; outline-style: dotted;
outline-color: #00dd00;
outline-width: medium;
- canvi de color d'enllaços visitats
a.intern {
color: #bb0000;
}
a.intern:visited {
color: #9b6a5b;
}
- afegir imatge al final
.nou:after
{
content: url(im/nou.png);
}
- tooltips
- How
TO - Tooltip
<div
class="tooltip">Hover over me
<span class="tooltiptext">Tooltip
text</span>
</div>
|
XHTML
|
|
|
XML
|
- Extensible Markup Language
(XML) (W3C)
- Schemas
- The XML library for Gnome
- <recursos XML/>
- XML Pitstop
- Search
- Validate
- Style
- XSL (EXtensible Stylesheet Language)
- style sheets for XML (CSS = style sheets for HTML)
- Transform
- Formats
- XSLT
-
-
- XML
and XSLT (w3schools)
-
- Conversion XML -> HTML
- browser will take file specified on
xml-stylesheet to render XML as HTML; if no
xml-stylesheet is specified, it will render raw
xml
-
| cdcatalog.xml |
cdcatalog.xsl (xml -> html) |
cdcatalog_text.xsl (xml -> txt) |
<?xml version="1.0"
encoding="UTF-8"?>
<?xml-stylesheet
type="text/xsl"
href="cdcatalog.xsl"?>
<catalog>
<cd>
<title>Empire
Burlesque</title>
<artist>Bob
Dylan</artist>
<country>USA</country>
<company>Columbia</company>
<price>10.90</price>
<year>1985</year>
</cd>
.
.
</catalog> |
<?xml version="1.0"
encoding="UTF-8"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output
method="html"/>
<xsl:template
match="/">
<html>
<body>
<h2>My CD
Collection</h2>
<table border="1">
<tr
bgcolor="#9acd32">
<th>Title</th>
<th>Artist</th>
</tr>
<xsl:for-each
select="catalog/cd">
<xsl:sort
select="artist"/>
<tr>
<td><xsl:value-of
select="title"/></td>
<td><xsl:value-of
select="artist"/></td>
</tr>
</xsl:for-each>
</table>
</body>
</html>
</xsl:template>
</xsl:stylesheet> |
<?xml version="1.0"
encoding="UTF-8"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="text"
media-type="text/plain"/>
<xsl:template match="/">
<xsl:for-each
select="catalog/cd">
<xsl:value-of select="title"/>
<xsl:text>: </xsl:text>
<xsl:value-of select="artist"/>
<xsl:text>
</xsl:text>
</xsl:for-each>
</xsl:template>
</xsl:stylesheet> |
- Conversion XML -> JSON
-
- Tools
- Exemples / Examples
- Editors
- XML
Editing (openlaszlo)
- Emacs
- XAE
- modes
- nXML
(*)
- schemas must be in RelaxNG format (.rnc) (conversion from XSD)
- add lines to file
.../nxml-mode-20041004/schema/schemas.xml:
<namespace
ns="urn:mpeg:mpeg4:SAF:2005"
typeId="SAF"/><typeId id="SAF"
uri="saf2.rnc"/>
~/.emacs
- (load
".../nxml-mode-20041004/rng-auto.el")
(setq auto-mode-alist
(cons
'("\\.\\(xml\\|xsl\\|rng\\|xhtml\\|xsr\\)\\'"
. nxml-mode)
auto-mode-alist))
- Eclipse
- Help -> Install new software: search for "XML"
- XMLBuddy
- qxmledit
- Conglomerate
- QXmlEditor
- KXML
Editor (KDE)
- MlView
(gnome)
- Commercial
- From bash:
- Biblioteques
/
Libraries
- Compressió / Compression
- Eines / Tools
- Trucs / Tips
- Diferències /
Differences (text
diff)
|
JavaScript
|
- ECMAScript
-
- Documentation
- Tutorials
- Reference
- JavaScript Kit
- JavaScript
Gamelan
Directory
- DevEdge (Netscape)
- Sash
- JavaScript
Calendar
- Free
Cut-and-Paste
JavaScript
- Javascript PC
emulator (micro Linux)
- JSON
- Comparison to XML
- JavaScript Object Notation
JSON
- Search
- JSON
Schema
- Similar to xsd for
xml
- Documents (IETF)
- Aprenentatge / Learning
- Extensió / Extension
- Eines / Tools
- used by:
- OpenAPI
Initiative (OAI) (Swagger)
- "OpenAPI 3.0 uses an extended subset of JSON
Schema Specification Wright Draft 00 (aka Draft
5) to describe the data formats."
- Django Rest Framework: Schemas
- Exemples / Examples
- JSON Schema
Store
- paypal/api-standards/v1/schema/json
(PayPal) (draft-04)
- simple.schema.json
{
"$schema":
"http://json-schema.org/draft-07/schema#",
"definitions": {},
"type": "object",
"properties": {
"name": {
"title": "Name",
"type": "string"
},
"label": {
"title": "Label",
"type": "string",
"enum": ["abc","zxc"],
"options": {
"enum_titles": ["Abc","Zxc"]
}
}
},
"required": [
"name",
"label"
]
}
- JSON-LD
(Linking Data)
- Exemples / Examples
- IPTC
(International Press Telecommunications Council)
- schema.org
- used by:
- Google: Understand
how structured data works
- index.html
<script
type="application/ld+json">
{
"@context":
"https://schema.org",
"@type": "Organization",
"url":
"http://www.example.com",
"name": "Unlimited Ball
Bearings Corp.",
"contactPoint": {
"@type":
"ContactPoint",
"telephone":
"+1-401-555-1212",
"contactType":
"Customer service"
}
}
</script>
- Parsers
- Parsing json with sed and
awk
- How to parse JSON string via command
line on Linux (jq)
- Python
- command line
- jsawk
- resty
(script wrapper for curl)
- jq
- jq play
(interactive player)
- removal
of
nodes, by naming nodes to excise
- Ús / Usage
- Tutorial
- Manual
- Cookbook
jq [options] '<commands>'
file.jsonjq -f file.jq ...
- file.jq
<sequential_command_1>
|
<sequential_command_2> |
(<parallel_command_3.1>),
(<parallel_command_3.2>) |
...
cat file.json | jq [options]
'<commands>'echo $my_var | jq
[options] '<commands>'-
special
character
|
description
|
|
|
separate
chained commands
|
()
|
group
commands
|
,
|
separate
parallel commands
|
+
|
concatenate
strings
|
- Jq
to replace text directly on file (like sed
-i)
-
- AWS
- mostra el contingut del fitxer / show the
file content
jq '.' toto.json |
sponge toto.json
- mostra el contingut de la variable / show
the variable content
- multiple input files
- show file1
jq -s '.[0]' file1.json
file2.json
- show file2
jq -s '.[1]' file1.json
file2.json
- merge file1 and file2:
- mostra una línia per a cada entrada, amb
dos dels camps separats per un espai:
jq -r '.[] | [.field_1,
.field_2] | join(" ")'
-
-
{
"llista": [
{
"nom": "primer",
"valor": 1,
"preu": 11
},
{
"nom": "segon",
"valor": 2,
"preu": 22
}
]
}
- crea un array amb els valors d'un sol dels
camps:
jq '[.llista[] | .nom]'
toto.json
-
jq '[.llista[] |
.nom] | join(" ")' toto.json- ordenats:
jq '[.llista[] | .nom] |
sort | join(" ")' toto.json
jq '.llista[] | .nom' toto.json
| paste
-sd' '
-
- (?)
my_array=($(echo
$my_json | jq '.llista[] | []'jq
'.llista[] | .nom' toto.json |
paste -sd' '))
my_array=($(echo $my_json |
jq '.llista[] | .nom' | paste -sd'
'))my_array=($(jq '.llista[] |
.nom' toto.json | paste -sd' '))
- posa en una sola línia la clau i el seu
valor:
jq 'to_entries[] | [.key,
.value] | join(" ")' toto.json- i es posa en un
#!/bin/bash
input="{
\"el primer\": \"un u\",
\"el segon\": \"un dos\",
\"el tercer\": \"un tres\"
}"
declare -A my_array
while IFS= read -r element
do
echo "-- read
element from jq: $element"
key=$(echo
"$element" | awk 'BEGIN{FS="#"}
{print $1}')
value=$(echo
"$element" | awk 'BEGIN{FS="#"}
{print $2}')
my_array["$key"]="$value"
done < <(echo "$input" | jq
-r 'to_entries[] | [.key, .value]
| join("#")')
for key in "${!my_array[@]}"
do
echo "${key}:
${my_array[$key]}"
done
exit 0
- create json
jq -n '{foo:"bar",foo2:3}'
jq -n '.foo="bar"'
jq -n '.[0].foo=1'
- variables bash
- Afegeix entrades a una llista / Add
entries to a list
part_1=$(jq -n
'[{foo:"bar",foo2:3}]')
part_2=$(jq -n
'[{foo:"bar2",foo2:4}]')
part_1=$(echo $part_1 | jq ". +=
$part_2")
echo $part_1 | jq '.'
- Substitució / Replacement
- Manipulate json as a bash variable:
body=$(jq -n
'{foo:"bar",foo2:3}')
echo "${body}" | jq '.'
body=$(echo "${body}" | jq '. +=
{foo3:"bar3"}')
echo "${body}" | jq '.'
value=bar4
body=$(echo "${body}" | jq ". +=
{foo3:\"${value}\"}")
echo "${body}" | jq '.'
object=$(jq -cr -n
'{foo4:"bar4",foo5:5}')
echo "${object}" | jq '.'
body=$(echo "${body}" | jq ". +=
{foo6:${object}}")
echo "${body}" | jq '.'
- Subconjunt / Subset
jq
'{field1:.field1,field2:.field2}'
- comprova si existeix my_key / check if
my_key exists
if
has("my_key") then .my_key else empty
end
#
desaconsellat, perquè si la clau
existeix is és un booleà fals es
complirà l'if i posarà empty
my_value=$(cat toto.json | jq -r 'if
.my_key then .my_key else empty end')
if ! [ "$my_value" ]
then
exit 1
fi
- if "my_key" exists, output "
-my_key
<my_key_value>"
(.my_key |
values | tostring | "-my_key "+.),
- fallback (alternative operator)
- si el valor és null, retorna una
cadena buida
my_value=$(echo
${curl_response} | jq '.my_key //
empty')
- si el valor és null, retorna una
cadena emmagatzemada en una variable de
la shell:
my_fallback_value="toto"
my_value=$(echo
${curl_response} | jq ".my_key //
${my_fallback_value}")
- filters
- map
-
".nginx.applications[] | select(.name
==\"$application_name\") | .record"- Només la primera / Only the first
occurrence:
".nginx.applications[] |
map( select(.name
==\"$application_name\") |
.record) | if .[0] then .[0] else
empty end"
- select elements with
name="my_prefix...":
'.my_list[] | select(.name |
startswith("my_prefix") )'
- boolean
select(... and ...)select(... or ...)
- key and value
- Select
objects based on value of variable
in object using jq
- Filter
objects based on tags in an array
- given the following json:
[
{
"title": "first",
"number": 1,
"tags": [
{"key": "foo1","value":
"bar1"},
{"key": "foo2","value":
"bar2"}
]
},
{
"title": "second",
"number": 2,
"tags": [
{"key": "foo1","value":
"bar2"},
{"key": "foo2","value":
"bar1"}
]
},
{
"title": "third",
"number": 3,
"tags": [
{"key": "foo1","value":
"bar1"}
]
}
] - select elements with a tag
foo1/bar1 (first, third, but not
second):
jq '.[] | select(
.tags[] | . and
.key=="foo1" and
.value=="bar1")'
-
-
- sort
(descending) by sum of several fields
[
.[] | ([.components[] | .bandwidth] |
add ) as $total_bandwidth |
.+{tbw:$total_bandwidth}
] |
sort_by(-.tbw)
- sort and swap first and second elements of
an array
- counter
- aplana la jerarquia de claus / flatten key
hierarchy
- convert json to ffmpeg parameters
.[] |
[
("-vsync vfr"),
(.components[] |
if .type!="image"
then
(if
.type=="video" then "v" else "a" end)
as $tipus |
#.lang
"-c:"+$tipus, .codec,
(if
.codec=="aac" then "-strict -2" else
empty end),
(.preset | values | if (. | length)
> 0 then "-preset "+. else empty
end),
#(.preset | values | "-"+$tipus+"pre
"+.),
"-b:"+$tipus, (.bandwidth | tostring),
(.channels | values | tostring | "-ac
"+.),
(if
.width then "-vf scale=w="+(.width |
tostring)+":h="+(.height | tostring)
else empty end),
#(if
.width then "-vf scale=w="+(.width |
tostring)+":h="+(.height |
tostring)+":force_original_aspect_ratio=decrease"
else empty end),
#(if
.width then "-vf
scale='-2:ceil(min(1\\,min("+(.width |
tostring)+"/iw\\,"+(.height |
tostring)+"/ih))*ih)'" else empty
end),
(.gop |
values | tostring | "-g "+.),
(.profile | values | if (. | length)
> 0 then "-profile:"+$tipus+" "+.
else empty end),
#(.profile | values | if (. | length)
> 0 then "-profile "+. else empty
end),
(.level
| values | tostring | "-level "+.),
(.keyint_min | values | tostring |
"-keyint_min "+.),
(.sc_threshold | values | tostring |
"-sc_threshold "+.)
else
empty
end
),
(.segment_duration | tostring |
"-hls_time "+.),
(.segment_list_size | tostring
| "-hls_list_size "+.),
(.segment_wrap | tostring |
"-hls_wrap "+.),
(.segment_start_number |
tostring | "-start_number "+.),
.name + ".m3u8"
]
| join(" ")
-
cat toto.json | python -m json.tool
- Documentation
- Conversion
- Typson
(TypeScript -> json-schema)
- Debugging
Javascript
- JSONP (wp)
- Toolkits, Frameworks
- Comparison
of
Javascript frameworks
- CAAT
(multi-instance director-based scene-graph manager)
(animacions 2D)
- APE project
(Ajax Push Engine)
- mootools
- Node.js
-
|
name
|
info
|
installation
|
usage
|
| Node.js |
|
Installation of
Node.js on system
|
System:
- Install
NodeJS (Angular JS 1: Tutorial)
- Mageia
urpmi nodejs (also
installs npm)
- CentOS
- Debian / Ubuntu
sudo apt-get install
nodejs-legacy
|
|
| nvm |
Node.js version
management:
| |
