Internet / WWW
|
Index
|
|
General
|
|
|
|
|
Aplicacions / Applications
|
|
|
|
CMS (content management)
|
|
Correu electrònic / E-mail
|
|
Dominis / Domains
|
|
Allotjament / Hosting
|
|
alta domini
|
renovació (¤/any)
|
|
emmagatzematge
|
tràfic
|
correu
|
altres
|
preus
|
|
|
|
|
|
|
comptes
|
espai
|
MySQL |
PHP
|
One Click
|
mensual
|
anual
|
active24
|
8
|
16,99
8
|
Basic Hosting
|
5GB
|
il·limitat
|
|
|
x
|
x
|
x
|
4,99
1,99
|
|
swhosting
|
8
|
17,75
|
Petit L
|
2GB
|
5GB/mes
|
|
|
-
|
x
|
-
|
3,50
|
|
Don
Dominio (.cat)
|
9,95
|
22,95 17,95
|
Pla
Mini
|
100MB
|
5GB/mes |
5
|
50MB
|
-
|
-
|
-
|
0,08
|
1 |
Pla
Basic
|
1GB
|
12GB/mes
|
100
|
300MB
|
100MB
|
x
|
|
2,08
|
25
|
Dina
Hosting
|
8,95
|
18,60
|
Hosting
Personal
|
2GB
|
30GB/mes
|
20
|
|
|
|
|
4,50
|
|
Virtual
Pyme
|
8,95
|
18,95
|
Mínim
|
1GB |
5GB/mes |
10
|
|
|
|
|
2,45
|
|
CD
Mon
|
8
|
19,95
|
Pla
Start
|
1GB
|
10GB/mes
|
1
|
|
-
|
x
|
|
2,50
|
30
|
Entorno
Digital
|
8
|
19,99
|
Pla
Presència
|
2GB
|
10GB/mes |
5
|
|
-
|
-
|
|
3,80
|
|
|
Feeds
|
|
Geolocalització /
Geolocation
|
|
Grups de notícies / Newsgroups
|
|
Gestors de preferits /
Bookmarks managers
|
|
|
|
Internet Services Providers (ISP)
|
|
Internet TV
|
|
Internet of things
|
|
|
|
|
- TCP
and
UDP port numbers (Wikipedia)
- Internet Protocol
(IP)
- W3C
- HTTP
- HTTPS
- Securing
the web
- Servidors / Servers
- Clients
- HSTS
- Let's
Encrypt
- wildcards
- Clients
- certbot
- Documentation
- Installation
- Compilation
- Requirements
- CentOS
sudo yum install
python2-mock
python-zope-interface python-zope-component
python-six python-setuptools
python2-acme
- Mageia
- Compile
certbot (recommended)
- or get it:
wget
https://dl.eff.org/certbot-auto
chmod a+x
certbot-auto
./certbot-auto
-debug
./certbot-auto
certonly
- Download
git clone
https://github.com/certbot/certbot.git
- Compile
cd certbot
sudo python setup.py install
- Compile
version >=0.22 (for wildcards)
cd certbot
virtualenv env
source env/bin/activate
pip install --upgrade pip
pip install --upgrade
setuptools
cd acme
python setup.py install
cd ..
python setup.py install
sudo ./env/bin/certbot ...
- Configuració / Setup
- serveis
amb usuari no root / non-root services
- How
to use certs in non-root services?
- letsencrypt_change_permissions.sh
#!/bin/bash
# file permissions
chgrp ssl-cert -R
/etc/letsencrypt
chmod 2755 /etc/letsencrypt
chmod g=rX -R /etc/letsencrypt
groupadd --gid 3000 ssl-cert
change_letsencrypt_persmissions.sh
usermod -a -G ssl-cert nginx
- crontab
certbot -n renew
--deploy-hook
/usr/local/bin/letsencrypt_change_permissions.sh
- Usage
|
info
|
usage
from
|
additional
setup for running server
|
|
|
CLI
options |
/etc/letsencrypt/renewal/your.domain.org.conf
|
Apache |
Nginx |
|
|
certbot
...
|
[renewalparams]
|
/etc/httpd/httpd.conf
|
/etc/nginx/nginx.conf
|
authentication
|
when
your html pages are on a static
server (hosted, AWS S3...)
|
--authenticator
manual
|
|
|
|
when
you have no running server, but
ports 80 and 443 are available. A
mini server will be started just for
the process.
E.g.: Postfix
email server
|
--authenticator
standalone |
authenticator
=
standalone
|
|
|
when
you have a running server (certbot
will generate some content locally,
in /path/to/.well-known/, which must
be available by http from
letsencrypt servers; this requires
additional setup of your server)
Can also be used to obtain a
certificate for Postfix
(-d mail.domain.org )
when an http server is already
running (even if it is running
with server_name
your.domain.org ). But make
sure that you have a DNS entry: mail.domain.org
CNAME your.domain.org |
--authenticator
webroot
-w /path/to
|
authenticator
=
webroot
[[webroot_map]]
your.domain.org = /path/to
|
(needed?)
<VirtualHost *:80>
Alias
/.well-known /path/to/.well-known
<Directory
"/path/to/.well-known">
Options Indexes FollowSymLinks
MultiViews
AllowOverride All
Require all
granted
</Directory>
|
server
{
listen
80;
server_name
your.domain.org;
# letsencrypt
location
/.well-known {
alias /path/to/.well-known;
}
|
when
you have a running Apache server |
--authenticator
apache |
authenticator
=
apache
|
|
|
when
you have a running Nginx server |
--authenticator
nginx |
authenticator
=
nginx
|
|
|
|
|
|
|
|
|
installation
|
you
do not want to install the obtained
certificate
|
certonly
|
installer
=
None
|
|
|
you
want to install the obtained
certificate in the Apache config
|
--installer
apache
|
installer
=
apache
|
|
|
you
want to install the obtained
certificate in the Nginx config |
--installer
nginx
|
installer
=
nginx
|
|
|
- Examples
- Just retrieve a certificate (a
nginx server is running):
webroot=/usr/share/nginx/html
letsencrypt_port=80
key_size=4096
email=me@my_company.com
domain=my_subdomain.my_company.com
sudo certbot certonly -n
--agree-tos --webroot -w
$webroot --http-01-port
$letsencrypt_port
--rsa-key-size $key_size
--email $email -d $domain
- just retrieve a wildcard
certificate (requires certbot
version >=0.22)
- install the required dns
plugin
sudo -i letsencrypt_port=80
key_size=4096
email=me@my_company.com
domain='*.my_company.com'
certbot certonly -n
--agree-tos --dns-route53
--http-01-port
$letsencrypt_port
--rsa-key-size $key_size
--email $email -d $domain --server
https://acme-v02.api.letsencrypt.org/directory
- Problemes / Problems
The currently
selected ACME CA endpoint
does not support issuing
wildcard certificates.
- Solució / Solution
--server
https://acme-v02.api.letsencrypt.org/directory
- Renewal
- Renewing
certificates
--pre-hook
--post-hook
--deploy-hook
- all certificates:
- specified certificates:
certbot certonly -n
--authenticator webroot
--webroot-path /path/to -d
your.domain.org
- Client
options
- Apache
- first time
certbot run --apache --http-01-port
80 --rsa-key-size 4096 --email your@email.org
-d your.domain.org
- non interactive:
certbot run --apache -n
--agree-tos --http-01-port
80
--rsa-key-size 4096 --email your@email.org
-d your.domain.org
- first renewal (because apache authenticator is
not working for certain apache config)
certbot certonly -n --authenticator
webroot --webroot-path /path/to/ -d
your.domain.org
- next renewals
sudo certbot -n renew
- Problemes / Problems
/etc/letsencrypt/options-ssl-apache.conf
not found
- Solució / Solution
sudo yum install
python2-certbot-apache
ln -s
/usr/lib/python2.7/site-packages/certbot_apache/options-ssl-apache.conf
/etc/letsencrypt/options-ssl-apache.conf
Could not open configuration
file
/etc/letsencrypt/options-ssl-apache.conf:
Permission denied
- This happens because
/etc/letsencrypt is a symbolic link
to /mnt/nfs/letsencrypt, a
nfs-mounted point
- Solució / Solution
- Nginx (using webroot)
- first time
- certbot
certonly --webroot -w /path/to
--http-01-port 80 --rsa-key-size 4096
--email your@email.org -d your.domain.org
- non interactive:
- certbot
certonly -n
--agree-tos --webroot -w
/path/to --http-01-port
80--rsa-key-size 4096 --email
your@email.org -d your.domain.org
- renewal
- /etc/nginx/conf.d/my_site.conf
server {
# letsencrypt
location
/.well-known {
alias /path/to/.well-known;
}
...
}
- check that
/etc/letsencrypt/renewal/my.site.org.conf
contains:
...
[renewalparams]
authenticator = webroot
webroot_path = /path/to
installer = None
...
sudo certbot renew --post-hook
"systemctl restart nginx.service"
- crontab
certbot -q
-n renew --post-hook
"systemctl restart nginx.service"
- Certificate installation in hosting
services
- Get only the certificate (follow the
instructions)
domain=my_subdomain.my_company.org
sudo certbot certonly --authenticator
manual -d $domain
- Install it:
- DonDominio
- Alojamiento web / Subdominios
- CERTIFICADO:
xclip -selection clipboard
<
/etc/letsencrypt/live/${domain} /cert.pem
- CERTIFICADO:
CTRL-V
- CLAVE SSL:
xclip -selection clipboard
< /etc/letsencrypt/live/ ${domain}
/ privkey.pem
- CLAVE SSL:
CTRL-V
- CERTIFICADOS DE AUTORIDAD:
xclip -selection clipboard
< /etc/letsencrypt/live/ ${domain}
/ chain.pem
- CERTIFICADOS DE AUTORIDAD:
CTRL-V
- AWS
- if you want to use certificates for
LoadBalancer or CloudFront, consider using AWS ACM
instead
- EC2
- Elastic Load Balancer (ELB)
- Let's
encrypt
and ELBs?
- First time
- create a load balancer with a port 80
listener
- AWS IAM: grant permissions to
user/role that will be used in next
steps
elasticloadbalancing:CreateLoadBalancerListeners
iam:GetServerCertificate
iam:UploadServerCertificate
- Example of policy:
{
"Version":
"2012-10-17",
"Statement": [
{
"Sid":
"",
"Effect":
"Allow",
"Action":
[
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:CreateLoadBalancerListeners",
"elasticloadbalancing:SetLoadBalancerListenerSSLCertificate"
],
"Resource":
[
"*"
]
},
{
"Sid":
"",
"Effect":
"Allow",
"Action":
[
"iam:ListServerCertificates",
"iam:GetServerCertificate",
"iam:UploadServerCertificate"
],
"Resource":
[
"*"
]
}
]
}
- from an instance (e.g. from an
instance behind the load balancer)
- get a LE certificate (IMPORTANT:
keys with 4096 bytes are not
allowed; only 2048 or
1024 (not allowed by Let's
Encrypt))
certbot ... --rsa-key-size
2048 ...
- upload the certificate to IAM (Upload
the
Certificate):
aws iam
upload-server-certificate
--server-certificate-name
${domain}.cert \
--certificate-body
file:///etc/letsencrypt/live/${domain}/cert.pem
\
--private-key file:// /etc/letsencrypt /live/${domain}/privkey.pem
\
--certificate-chain file:// /etc/letsencrypt /live/${domain}/chain.pem
- Get the ARN of the uploaded
certificate
ARN=$(aws --output json
iam get-server-certificate
--server-certificate-name
${domain}.cert | jq
'.ServerCertificate.ServerCertificateMetadata.Arn')
- Add
an
HTTPS Listener Using the AWS CLI
aws elb create-load-balancer-listeners
--load-balancer-name
my-load-balancer --listeners
Protocol=HTTPS,LoadBalancerPort=443,InstanceProtocol=HTTPS,InstancePort=443,SSLCertificateId=$ARN
- ...
- Renewal
- letsencrypt-aws
- Installation
git clone
https://github.com/alex/letsencrypt-aws.git
cd letsencrypt-aws
virtualenv env
source env/bin/activate
pip install -r
requirements.txt
deactivate
- First time
- ask ACME for a private key
source
env/bin/activate
python
letsencrypt-aws.py
register email@host.com
deactivate
- save it as
email_host_com.privkey.acme.pem
(locally or on AWS S3)
- create an IAM
policy (name it e.g.
LetsencryptAWS)
- Problemes / Problems
- The
requested
nginx plugin does not appear to be installed?
#1736
- Solution: manual request and installation
cd letsencrypt
./letsencrypt-auto certonly -a
manual --rsa-key-size 4096 --email
your@email.org -d your.domain.org
- /etc/nginx/nginx.conf (or
/etc/nginx/conf.d/your_site.conf)
ssl_certificate
/etc/letsencrypt/live/www.example.org/fullchain.pem;
ssl_certificate_key
/etc/letsencrypt/live/www.example.org/privkey.pem;
- Note: if you specify cert.pem instead of
fullchain.pem, browsers will work, but curl nor openssl s_client
won't:
- Content
negotiation
- JPEG JFIF
- SOAP (Simple
Object Access Protocol)
- Platform for Privacy
Preferences (P3P) (press
release)
- Resource Description Framework (wp)
(metadades/metadata)
- Media
Fragments URI (MFWG)
- WebVTT
(Web Video Text Tracks)
- Cross-Origin
Resource Sharing (CORS)
Access-Control-Allow-Origin:
http://foo.example/
- W3C TV
- IETF RFC
- Protocols de missatgeria /
Messaging protocols
-
- AMQP (Advanced Message Queuing Protocol)
(wp)
- MQTT (Message
Queue Telemetry Transport) (wp)
- Brokers
- Clients
- Android
- CLI
- mqtt
(Mosquitto)
mosquitto_pub (producer)
mosquitto_sub (consumer)
- rtl_433
rtl_433 -F "mqtt://localhost:1883"
rtl_433 -F
"mqtt://localhost:1883,retain=1"
- web
- MQTT.fx
(JavaFX)
- Instal·lació / Installation Download
- rpm
wget
http://www.jensd.de/apps/mqttfx/1.7.1/mqttfx-1.7.1-1.x86_64.rpm
- Mageia
urpmi
mqttfx-1.7.1-1.x86_64.rpm
- Ús / Usage
- Python
- Other
- Projects
- STOMP
(Streaming Text Oriented Messaging Protocol) (wp)
- to connect to message-oriented
middleware
- STOMP
Protocol
Specification, Version 1.2
|
frame |
standard
headers
|
request headers |
response headers |
|
|
content-length |
content-type |
receipt |
|
|
connection
frames |
CONNECT |
MAY |
|
|
accept-version
host
login
passcode
heart-beat
|
|
CONNECTED |
MAY |
|
|
|
version
heart-beat
session
server
|
client
frames |
SEND |
MAY |
SHOULD |
MAY |
|
|
SUBSCRIBE |
MAY |
|
MAY |
destination
id
ack: [auto, client,
client-individual]
|
|
UNSUBSCRIBE |
MAY |
|
MAY |
|
|
BEGIN |
MAY |
|
MAY |
|
|
COMMIT |
MAY |
|
MAY |
|
|
ABORT |
MAY |
|
MAY |
|
|
ACK |
MAY |
|
MAY |
|
|
NACK |
MAY |
|
MAY |
|
|
DISCONNECT |
MAY |
|
MAY |
|
|
server
frames |
MESSAGE |
MAY |
SHOULD |
|
|
destination
message-id
subscription
ack
|
RECEIPT |
MAY |
|
|
|
|
ERROR |
MAY |
SHOULD |
|
|
|
- Server
- Client (producer: SEND; consumer: SUBSCRIBE)
- RadioDNS
- Notes
- frame
COMMAND
header1:value1
header2:value2
Body^@
telnet vis.muscradio.com 61613
Trying 81.20.48.151...
Connected to 81.20.48.151.
Escape character is '^]'.
CONNECT
^@
SUBSCRIBE
destination: /topic/fm/ce1/c479/09580/image
ack: auto
^@
DISCONNECT
receipt:77
^@
- HTTP
headers
- HTTP
request
methods (wp)
-
method
|
description
|
note
|
usage
|
related HTML tag
|
|
|
|
safe
|
idempotent
|
|
HEAD
|
same as GET, but
without the body
|
|
|
|
|
GET
|
requests a
representation of the resource
|
only to retrieve
|
y
|
y
|
href
|
POST
|
submits data to be
processed
|
|
n
|
n
|
form method="post"
|
PUT
|
uploads a
representation of the resource
|
|
n
|
y
|
|
DELETE
|
deletes the resource
|
|
|
|
|
TRACE
|
echoes the received
request
|
|
|
|
|
OPTIONS
|
http methods
supported by the server
|
|
|
|
|
CONNECT
|
|
|
|
|
|
PATCH
|
apply partial
modifications to a resource |
|
|
|
|
- Ús / Usage
- curl -X
http_method
- Django
- POST
-
generation
|
received
by CGI
|
html form
|
curl
|
content-type
|
as standard input
|
<form
action="my_cgi.sh" method="post">
<input type="text" name="tata"
value="tata_value">
<input type="text" name="titi"
value="titi_value">
<input type="submit">
</form>
|
curl -i -X POST --data
'tata=tata_value' --data 'titi=titi_value'
curl -i -H
"Content-Type:
application/x-www-form-urlencoded"
-d
'tata=tata_value&titi=titi_value'
curl -i -X POST --data-binary
'tata=tata_value' --data-binary
'titi=titi_value'
|
application/x-www-form-urlencoded
|
tata=tata_value&titi=titi_value
|
<form
action="my_cgi.sh" method="post" enctype="multipart/form-data">
<input type="text" name="tata"
value="tata_value">
<input type="text" name="titi"
value="titi_value">
<input type="submit">
</form> |
curl -i -X POST -F
tata=tata_value -F titi=titi_value
curl -i -X POST -H
'Content-Type: multipart/form-data'
-F
tata=tata_value -F titi=titi_value
|
multipart/form-data;
boundary=------------------------6e0b5ea0578a6399
|
--------------------------6e0b5ea0578a6399
Content-Disposition: form-data; name="tata"
tata_value
--------------------------6e0b5ea0578a6399
Content-Disposition: form-data; name="titi"
titi_value
--------------------------6e0b5ea0578a6399--
|
|
curl -i -X POST -H
'Content-Type: application/json'
--data
'{"toto":"toto_value","tata":"tata_value"}'
url="https://127.0.0.1:8000/"
my_var="toto value";
json_code="{\"toto\":\"${my_var}\"}";
curl_options=" -H
'Content-Type: application/json'
--data '$json_code' ";
curl_command="curl -v -X POST
$curl_options '$url'";
eval "$curl_command"
my_var="toto l'educat"; my_escaped_var=$(echo
"${my_var}" | sed
"s/'/'\\\''/g");
json_code="{\"toto\":\"${my_escaped_var}\"}";
curl_options=" -H
'Content-Type: application/json'
--data '$json_code' ";
curl_command="curl -v -X POST
$curl_options '$url'";
eval "$curl_command"
|
application/json
|
|
- HTTP status codes
- Response
Status Codes (RFC 7231, Section 6)
- Hyper Text
Coffee Pot Control Protocol (HTCPCP/1.0) (RFC 2324)
- List of HTTP status codes (wp)
- Setting
HTTP
Status Codes (Servlet tutorial)
-
family
|
description
|
code |
1xx
|
Informational
|
- 100 Continue
- 101 Switching Protocols
- 102 Processing
- 103 Early Hints
|
2xx
|
Success
|
- 200 OK
- 201 Created
- 202 Accepted
- ...
|
3xx
|
Redirection
|
- 300 Multiple Choices
- 301 Moved Permanently (POST
is switched to GET)
- ...
- 308 Permanent Redirect (do not switch methods)
|
4xx
|
Client error
|
- 400 Bad Request
- 401 Unauthorized
- 402 Payment Required
- 403 Forbidden
- 404 Not Found
- 405 Method Not Allowed
- ...
- 408 Request Timeout
- ...
- 409 Conflict
- ...
- 415 Unsupported Media Type
- ...
- 418 I'm a teapot
- ...
- 429 Too Many Requests
- ...
- 451 Unavailable For Legal Reasons
|
5xx
|
Server error
|
- 500 Internal Server Error
- 501 Not Implemented
- 502 Bad Gateway
- 503 Service Unavailable
- 504 Gateway Timeout
- 505 HTTP Version Not Supported
- ...
|
- 4xx
- DOCTYPE
/ DTD
- Emmagatzematge / Storage
- Same
origin policy (wp)
- Push
- Comet / Ajax push (wp)
- Categories
- Streaming
- Ajax with long polling
- Implementations
- WebSocket (wp)
(HTML5)
- Media
types
(ftp)
- Byte articles
- Cache
- vCard and vCalendar
- MIME-Types
- IANA
MIME Media Types
- HTTP
servers configuration
/etc/apache2/apache2.conf (or /etc/apache2/sites-available/my_config)
AddType
application/dash+xml .mpd .xml
- /etc/httpd/conf/mime.types
- IIS
- HTTP headers / File types
- Imatges / Images
|
|
- General
- Articles
- Programari / Software
- Server
- Client
- Discover
|
|
- Comparison
of
web servers (wp)
- Comparison
of
lightweight web servers (wp)
- Best
Practices
for Speeding Up Your Web Site (Steve Sounders)
- Servidors HTTPS / HTTPS
server
- Balanceig de càrrega / Load
balancing
- Django
deployment: scaling
- Solucions / Solutions
- DNS
- Maquinari / Hardware
- Programari / Software
- Cache on servers
- Servidors
de prova / Test servers
- Nginx (nginx.com) (wp)
- Compilació /
Compilation
- Init scripts
- show compilation options
- Instal·lació / Installation
- Mageia
urpmi nginx
sytemctl enable nginx
sytemctl start nginx
- Ubuntu
- CentOS
- How
To
Install Nginx on CentOS 7
- LEMP
server
on CentOS 7 with FastCGI
sudo yum install epel-release
sudo yum install nginx
- Official
nginx repositories for CentOS
- /etc/yum.repos.d/nginx.repo
sudo systemctl start nginx.service
- Firewall
(only needed if firewalld.service is running)
sudo firewall-cmd --permanent
--zone=public --add-service=http
sudo firewall-cmd --permanent --zone=public
--add-service=https
sudo firewall-cmd --reload
- SELinux
- SELinux
on HTTP servers
sudo setsebool -P
httpd_read_user_content 1
- if only using nginx (not uwsgi)
- if using also uwsgi (unix socket permission
denied)
- SELinux
policy
for nginx and GitLab unix socket in Fedora
19
- disable Selinux:
- detect the problem
- access your website, to generate logs
sudo yum install
policycoreutils-python
sudo grep nginx
/var/log/audit/audit.log | audit2allow
- solve the problem (do not use -M option,
as you will not be able to edit the te text
file to add rules)
sudo -i
grep nginx
/var/log/audit/audit.log | audit2allow
-m nginx > nginx.te
checkmodule -M -m -o nginx.mod
nginx.te
semodule_package -o
nginx.pp -m nginx.mod
semodule -i nginx.pp
setenforce 1
- example
- nginx.te
module
nginx 1.0;
require {
type httpd_t;
type init_t;
type nfs_t;
type user_home_t;
type var_lib_t;
type unlabeled_t;
class unix_stream_socket
connectto;
class file {read getattr open};
class lnk_file read;
class sock_file write;
}
#============= httpd_t
==============
allow httpd_t
init_t:unix_stream_socket
connectto;
allow httpd_t nfs_t:file {read
getattr open};
allow httpd_t nfs_t:lnk_file read;
allow httpd_t user_home_t:file
{read open};
allow httpd_t var_lib_t:sock_file
write;
allow httpd_t unlabeled_t:lnk_file
read;
allow httpd_t unlabeled_t:file
{read open};
- Problems
/ Problemes
- Django wsgi
- Documentation
(.org)
- Beginner's
Guide
- Modules
-
|
|
nginx.org
|
nginx.com |
|
|
|
3rd party
|
core
|
|
ngx_http_core_module
|
|
|
|
add_header (any header)
expires (cache headers)
-
|
examples |
HTTP headers |
add_header |
|
|
expires |
# at now +
time (>=0)
expires 1d; |
Date: Thu, 23
Apr 2020 15:34:53 GMT
Last-Modified: Thu, 23 Apr
2020 15:19:47 GMT
Expires: Fri, 24 Apr
2020 15:34:53 GMT
Cache-Control: max-age=86400
|
# at now + time (<0)
expires -1d; |
Date: Thu, 23 Apr 2020
15:36:14 GMT
Last-Modified: Thu, 23 Apr
2020 15:19:47 GMT
Expires: Wed, 22 Apr 2020
15:36:14 GMT
Cache-Control: no-cache |
# at file modification +
time
expires modified 1d; |
Date: Thu, 23 Apr 2020
15:37:35 GMT
Last-Modified: Thu,
23 Apr 2020 15:19:47 GMT
Expires: Fri, 24 Apr
2020 15:19:47 GMT
Cache-Control: max-age= seconds_remaining_to_23_Apr_2020:15:19:47_GMT
|
# at precise date (local
time)
expires @15h30m; |
Date: Thu, 23 Apr 2020
15:39:34 GMT
Last-Modified: Thu, 23 Apr
2020 15:19:47 GMT
Expires: Fri, 24 Apr 2020 13:30:00
GMT
Cache-Control: max-age=seconds_remaining_to_24_Apr_2020:13:30:00_GMT
|
# at 1st Jan 1970
expires epoch; |
Expires: Thu,
01 Jan 1970 00:00:01 GMT
Cache-Control: no-cache
|
# at 31st Dec 2037
(max-age: 10 years)
expires max; |
Expires: Thu,
31 Dec 2037 23:55:55 GMT
Cache-Control:
max-age=315360000
|
# no cache headers
expires off; |
Expires
Cache-Control
|
|
|
|
|
ngx_http_rewrite_module
|
|
proxy
|
|
ngx_http_proxy_module
|
|
fastCGI
|
|
ngx_http_fastcgi_module
|
|
authentication |
Access
|
ngx_http_access_module
|
|
Basic
|
ngx_http_auth_basic_module
|
|
JWT
|
ngx_http_auth_jwt_module
|
|
Digest
|
|
ngx_http_auth_digest
|
PAM
|
|
ngx_http_auth_pam_module
|
Request
|
ngx_http_auth_request_module
|
ngx_http_auth_request_module
|
languages
|
LUA
|
|
ngx_http_lua_module |
...
|
|
|
|
- Service setup
- NGINX
systemd service file
nginx.service
# https://www.nginx.com/resources/wiki/start/topics/examples/systemd/
[Unit]
Description=The NGINX HTTP and reverse proxy
server
After=syslog.target network.target
remote-fs.target nss-lookup.target
cloud-init.service
[Service]
Type=forking
PIDFile=/run/nginx.pid
TimeoutStartSec=200s
ExecStartPre=/usr/sbin/nginx -t
PrivateTmp=true
# other ExecStartPre scripts
ExecStartPre=/bin/bash
-c 'for script in /etc/nginx/execstartpre.d/*;
do $script; done'
ExecStart=/usr/sbin/nginx
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
[Install]
WantedBy=multi-user.target
- /tmp
- /usr/lib/systemd/system/nginx.service
- real /tmp dir for nginx is:
/tmp/systemd-private-...-nginx.service-.../tmp
- Configuració /
Configuration
- How
To Optimize Nginx Configuration
- Modificació
del fitxer de configuració / Modification of config file
- Modular
- /etc/nginx/
- nginx.conf
user nginx;
worker_processes 1;
http {
include
/etc/nginx/mime.types;
default_type
application/octet-stream;
# Load modular
configuration files from the
/etc/nginx/conf.d directory.
# See
http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
} # nginx-rtmp-module
rtmp {
# Load modular
configuration files from the
/etc/nginx/rtmp.conf.d directory.
include /etc/nginx/rtmp.conf.d/*.conf;
}
- execstartpre.d/
- script1_execstartpre.sh
- ...
- conf.d/
- http_server_1.conf
server {
listen
80;
server_name
localhost;
# Load modular
configuration files from the
/etc/nginx/conf.d/server_1.d/
directory.
include
/etc/nginx/conf.d/http_server_1.d/*.http_server.conf;
}
- http_server_1.d/
- dir_1_a.http_server.conf
location
/dir_1_a {
alias
/path/to/dir_1_a;
}
- rtmp_server_1.http_server.conf (if
using
nginx-rtmp-module
statistics
and control)
# rtmp statistics
location /stat {
rtmp_stat all;
rtmp_stat_stylesheet stat.xsl;
}
location /stat.xsl {
root
/path/to/parent/dir/of/stat/xsl/file;
}
# rtmp control
location /control {
rtmp_control
all;
}
- https_server_2.conf
server {
# https
listen
443 ssl;
server_name
www.example.org;
ssl_certificate
/etc/letsencrypt/live/www.example.org/fullchain.pem;
ssl_certificate_key
/etc/letsencrypt/live/www.example.org/privkey.pem;
ssl_protocols
TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers
HIGH:!aNULL:!MD5;
...
}
- https_server_2.d/
- rtmp.conf.d
- rtmp_server_1.conf
server {
listen 1935;
ping 30s;
notify_method get;
# compatibility
with GStreamer:
publish_time_fix
off;
# Load modular
configuration files from the
/etc/nginx/rtmp.conf.d/wct-rtmp/
directory.
include
/etc/nginx/rtmp.conf.d/rtmp_server_1.d/*.rtmp_server.conf;
}
- rtmp_server_1.d/
- app_1_a.rtmp_server.conf
application
app_1_a {
live on;
wait_video on;
wait_key on;
}
- rtmp_server_2.conf
- rtmp_server_2.d/
- HTTPS
- Configuring
HTTPS
servers
- Client
Side
Certificate Auth in Nginx
- HSTS
- nginx.conf
server {
# https
listen
443 ssl;
server_name
www.example.org;
ssl_certificate
/etc/letsencrypt/live/www.example.org/fullchain.pem;
ssl_certificate_key
/etc/letsencrypt/live/www.example.org/privkey.pem;
#ssl_protocols
TLSv1 TLSv1.1 TLSv1.2;
ssl_protocols
TLSv1.2;
ssl_ciphers
HIGH:!aNULL:!MD5;
# hsts
add_header
Strict-Transport-Security "max-age=31536000;
includeSubDomains; preload" always;
...
- Autenticació /
Authentication
- Debug
- A
debugging log (nginx.org)
- Debugging log for selected clients
- Logging to a cyclic memory buffer
- Debugging
NGINX (nginx.com)
- when building
./configure --with-debug ...
- check it:
nginx -V 2>&1 | grep --
'--with-debug'
- nginx.conf
error_log
/path/to/log debug;
error_log
/var/log/nginx/error.log debug;
- level possible values:
debug, info, notice,
warn, error, crit, alert, emerg
- tail -n 200 -f
/var/log/nginx/error.log
- Monitoratge / Monitoring
- ngxtop
- Instal·lació / Installation
- Ús / Usage
- only last entries (live)
- all log content
ngxtop --no-follow -i 'status >=
500' print request status http_referer
- Headers
- your.conf
location /toto {
add_header x-peticio $request;
add_header x-temps $request_time;
add_header x-longitud $request_length;
return
200 'tot bé\n';
}
- Proxying to:
-
backend
server
|
start server
|
nginx pass config
|
FastCGI
|
PHP |
php-cgi -b
127.0.0.1:9000 |
location ~
\.php$ {
include
/etc/nginx/fcgi_params; #or whatever you named
it
fastcgi_pass
127.0.0.1:9000;
}
|
FCGIWrap |
systemctl
start spawn-fcgi |
fastcgi_pass
unix:/tmp/cgi.sock;
|
memcached |
|
|
|
ngx_http_proxy_module
|
|
|
proxy_pass
...
|
SCGI |
|
|
|
upload-module
|
|
|
upload_pass
...
|
UWSGI |
|
bin/uwsgi
|
uwsgi_pass
...
|
- FastCGI (CGI)
- fastcgi
module
(ngx_http_fastcgi_module)
- Understanding
and
Implementing FastCGI Proxying in Nginx
- FastCGI
example
- fastcgi_params
Versus
fastcgi.conf – Nginx Config History
- nginx
-
How to run a shell script on every request?
- FCGIWrap
(github)
- Documentation
- Installing
FcgiWrap
and Enabling Perl, Ruby and Bash Dynamic
Languages on Gentoo LEMP
- Installation
- CentOS
- from source
- 14
Install
Nginx, PHP5 (PHP-FPM), And
Fcgiwrap
- Serving
CGI
Scripts With Nginx On CentOS 6.0 -
Page 2
- Passos / Steps
- install dependencies
sudo yum -y install
gcc autoconf automake
sudo yum -y install
git
sudo yum -y install
fcgi-devel
- if you did not installed
nginx from source:
sudo yum -y
install
nginx-all-modules
- download and compile fcgiwrap
cd ~src
git clone
git://github.com/gnosek/fcgiwrap.git
cd fcgiwrap
autoreconf -i
./configure
make
sudo make install
- install
and setup spawn
fcgi
- install
sudo yum -y
install spawn-fcgi
- setup
- /etc/sysconfig/spawn-fcgi
#
You must set some
working options
before the
"spawn-fcgi"
service will work.
# If SOCKET points
to a file, then
this file is
cleaned up by the
init script.
#
# See
spawn-fcgi(1) for
all possible
options.
#
# Example :
#SOCKET=/var/run/php-fcgi.sock
#OPTIONS="-u
apache -g apache
-s $SOCKET -S -M
0600 -C 32 -F 1 -P
/var/run/spawn-fcgi.pid
--
/usr/bin/php-cgi"
FCGI_SOCKET=/var/run/fcgiwrap.socket
FCGI_PROGRAM=/usr/local/sbin/fcgiwrap
FCGI_USER=nginx
FCGI_GROUP=nginx
FCGI_EXTRA_OPTIONS="-M
0770"
OPTIONS="-u
$FCGI_USER -g
$FCGI_GROUP -s
$FCGI_SOCKET -S
$FCGI_EXTRA_OPTIONS
-F 1 -P
/var/run/spawn-fcgi.pid
-- $FCGI_PROGRAM"
- to redirect stderr
logs
to
/var/log/nginx/error.log
(they will be shown
as "
FastCGI
sent in stderr: ")
- ...
OPTIONS="-u
$FCGI_USER -g
$FCGI_GROUP -s
$FCGI_SOCKET
-S
$FCGI_EXTRA_OPTIONS
-F 1 -P
/var/run/spawn-fcgi.pid
--
$FCGI_PROGRAM
-f"
- start
systemctl enable
spawn-fcgi
systemctl start
spawn-fcgi
systemctl status
spawn-fcgi
- Problemes /
Problems
- Security issues
- Example to execute a script specified in the
url
- setup
- /etc/nginx/default.d/toto.conf
location
/cgi-bin/ {
# Disable
gzip (it makes scripts feel slower
since they have to complete
# before
getting gzipped)
gzip off;
# Set the
root to /usr/lib (inside this
location this means that we are
# giving
access to the files under
/usr/lib/cgi-bin)
root /usr/share/nginx;
# Fastcgi
socket
fastcgi_pass unix:/var/run/fcgiwrap.socket;
# Fastcgi
parameters:
include the standard ones
include /etc/nginx/fastcgi_params;
# Fastcgi
parameters:
adjust non standard
parameters (e.g. SCRIPT_FILENAME)
fastcgi_param
SCRIPT_FILENAME
$document_root$fastcgi_script_name;
# Fastcgi
parameters:
user-defined parameters
fastcgi_param
TOTO_PARAM $request_length;
}
sudo systemctl reload
nginx.service
- scripts
mkdir -p
/usr/share/nginx/cgi-bin
- perl script
- /usr/share/nginx/cgi-bin/hello_world.cgi
#!/usr/bin/perl
-w
#
Tell perl to send a html
header.
# So
your browser gets the output
#
rather then
<stdout>(command line
# on
the server.)
print "Content-type:
text/html\n\n";
#
print your basic html tags.
# and
the content of them.
print
"<html><head><title>Hello
World!!
</title></head>\n";
print
"<body><h1>Hello
world</h1></body></html>\n";
chmod 755
/usr/share/nginx/cgi-bin/hello_world.cgi
- bash
script
- /usr/share/nginx/cgi-bin/toto.sh
#!/bin/bash
# as PATH is set in
/etc/init.d/functions, you may
need to complete it
# when you are calling other
scripts (e.g. in
/usr/local/bin) from here
PATH=${PATH}:/usr/local/bin
export $PATH
# call a script in
/usr/local/bin
my_script.sh
# content type must be
present, followed by an empty
line
# all lines must be ended
with \r\n (CR LF)
echo -e "Content-type:
text/plain\r"
echo -e "\r"
# your text
echo "123"
echo "server software:
$SERVER_SOFTWARE"
echo "script filename:
$SCRIPT_FILENAME"
echo "toto param:
$TOTO_PARAM"
# print all available
environment variables
env
chmod 755
/usr/share/nginx/cgi-bin/toto.sh
- Debug
- /usr/share/nginx/cgi-bin/toto_debug.sh
#!/bin/bash
-x
# adding -x will make bash
xtrace output (by default
written to stderr) to
appear in
/var/log/nginx/error.log,
# if option -f was
specified in spawn
config
echo -e "Content-type:
text/plain\n"
# log to stderr instead of
response body
(1>&2 echo
"[hello]")
...
- Parse
GET and POST parameters (using cgi_functions.sh)
- See also CGI
bash examples with POST
- /usr/share/nginx/cgi-bin/toto_with_params
#!/bin/bash
# if
cgi_functions.sh is
installed in
/usr/local/bin
PATH=/usr/local/bin:${PATH}
export PATH=$PATH #
register all GET and POST
variables
source cgi_functions.sh
cgi_getvars BOTH ALL
1>&2
# content type must be
present, followed by an
empty line
# all lines must be ended
with \r\n (CR LF)
echo -e "Content-type:
text/plain\r"
echo -e "\r"
# variables from QUERY and
POST are available as
shell variables
echo "var1: $var1"
echo "var2: $var2"
echo "var3: $var3"
- GET
curl -X GET
http://127.0.0.1/cgi-bin/toto_with_params ?var1=value1
POST
- application/x-www-form-urlencoded
curl -X POST
--data 'var3=value3'
http://127.0.0.1/cgi-bin/toto_with_params
- multipart/form-data
curl -X POST
-F var2=value2
http://127.0.0.1/cgi-bin/toto_with_params ?var1=value1
curl -X POST
-F var2=value2
http://127.0.0.1/cgi-bin/toto_with_params
- from local:
REQUEST_METHOD=POST
QUERY_STRING="var1=value1&var4=value4"
var2=value2 var3=value3
/usr/share/nginx/cgi-bin/toto_with_params
- Allow only POST:
- /usr/share/nginx/cgi-bin/toto_only_post
#!/bin/bash
# register
all GET and POST variables
source cgi_functions.sh
cgi_getvars BOTH ALL
1>&2
if [[ $REQUEST_METHOD !=
"POST" ]]
then
echo
"Status: 405 Method Not
Allowed"
echo -e
"Content-type:
text/plain\r"
echo -e
"\r"
echo
"Method not allowed"
exit 0
fi
# content
type must be present,
followed by an empty line
# all
lines must be ended
with \r\n (CR LF)
echo -e "Content-type:
text/plain\r"
echo -e "\r"
...
- Client
- Problemes / Problems
- SELinux
- 403 Forbidden
- 502 Bad gateway
- check that script returns as first
lines (optionally a
Status
line, containing return code number
and reason
phrase, see: ngx_http_fastcgi_hide_headers)
(note the empty line):
Content-type:
text/plain
Status:
202 OK
Content-type: text/plain
- Example to execute a fixed script when
accessing a url:
- nginx
-
How to run a shell script on every
request?
- ...conf with standard parameters
location
/do_something/ {
# Disable
gzip (it makes scripts feel slower
since they have to complete
# before
getting gzipped)
gzip off;
# Set the
root to /usr/lib (inside this location
this means that we are
# giving
access to the files under
/usr/lib/cgi-bin)
root
/usr/share/nginx;
# Fastcgi
socket
fastcgi_pass unix:/var/run/fcgiwrap.socket;
# Fastcgi parameters,
include the standard ones
include /etc/nginx/fastcgi_params;
# Adjust non
standard parameters (SCRIPT_FILENAME)
fastcgi_param
SCRIPT_FILENAME
/path/to/fixed_script.sh;
#
user-defined parameters
fastcgi_param
TOTO_PARAM $request_length;
}
- fixed_script.sh (see also simple_cgi.sh
for an example with POST)
#!/bin/bash
# content type must be present,
followed by an empty line #
all lines must be ended with \r\n (CR LF)
echo -e "Content-type: text/plain\r"
echo -e "\r"
# execute some script
result=$(/path/to/do_something_script.sh)
echo "result:"
echo $result
# your text
echo "request method:
$REQUEST_METHOD"
echo "server software:
$SERVER_SOFTWARE"
echo "script filename:
$SCRIPT_FILENAME"
echo "toto param:
$TOTO_PARAM"
- fixed_script_with_specified http
response code
- conf with parameters from url
- location
/fast {
# Disable gzip (it
makes scripts feel slower since they
have to complete
# before getting
gzipped
gzip off;
# script will be
under
root
/usr/share/nginx;
# Fastcgi socket
fastcgi_pass
unix:/var/run/fcgiwrap.socket;
# Fastcgi
parameters, include the standard ones
include
/etc/nginx/fastcgi_params;
# script to be
executed
fastcgi_param
SCRIPT_FILENAME
/usr/share/nginx/cgi-bin/fixed_script.sh;
# parameters for
the script
fastcgi_param
REQUEST_URI $request_uri;
# parameters from
url
#
/fast/<param1>/<param2>/
if ($request_uri ~*
"/fast/([^/]*)/([^/]*)/$" ) {
set $param1 $1;
set $param2 $2;
}
fastcgi_param
PARAM1 $param1;
fastcgi_param
PARAM2 $param2;
}
- fixed_script with parameters
#!/bin/bash
# content type must be present,
followed by an empty line
# all lines must be ended with \r\n
(CR LF)
echo -e "Content-type: text/plain\r"
echo -e "\r"
echo "REQUEST_URI: ${REQUEST_URI}"
echo "PARAM1: ${PARAM1}"
echo "PARAM2: ${PARAM2}"
exit 0
- Example to get upload time:
- .../upload.conf
location /up {
# allow 100
Continue
client_max_body_size 20M;
# allow upload
sendfile on;
# Disable gzip (it
makes scripts feel slower since they
have to complete
# before getting
gzipped)
gzip off;
# Set the root to
/usr/lib (inside this location this
means that we are
# giving access to
the files under /usr/lib/cgi-bin)
root
/usr/share/nginx;
# Fastcgi socket
fastcgi_pass
unix:/var/run/fcgiwrap.socket;
# Fastcgi parameters,
include the standard ones
include /etc/nginx/fastcgi_params;
# Adjust non
standard parameters:
fastcgi_param
SCRIPT_FILENAME
/path/to/upload_time.sh;
# user-defined
parameters:
fastcgi_param
REQUEST_LENGTH $request_length;
fastcgi_param
REQUEST_TIME $request_time;
# user-defined
headers
add_header x-temps
$request_time;
add_header
x-longitud $request_length; }
- upload_time.sh
#!/bin/bash
# content type must be present,
followed by an empty line
# all lines must be ended with \r\n
(CR LF)
echo -e "Content-type: text/plain\r"
echo -e "\r"
# execute some script
result=$(/path/to/do_something_script.sh)
echo "result:"
echo $result
echo "request method:
$REQUEST_METHOD"
echo "request length:
$REQUEST_LENGTH"
echo "request time:
$REQUEST_TIME"
- Client
dd if=/dev/zero
of=megabyte.dat bs=1M
count=1
curl -i --data-binary
'@megabyte.dat'
http://<server>/up
- Cache
- CORS
- Upload
- Test upload speed
- Nginx
direct
file upload without passing them through backend
- nginx
upload
client_max_body_size issue
- client
- client_body_in_file_only
- files uploaded to
- ...conf
server {
listen 80;
server_name
localhost;
error_log
/var/log/nginx/error.debug.log debug;
#sendfile on;
location /up {
#auth_basic
"Restricted
Upload";
#auth_basic_user_file
basic.htpasswd;
#limit_except
POST
{ deny all; }
#client_body_temp_path
/tmp/;
#client_body_in_file_only on;
#client_body_buffer_size
128K;
# allow 100 Continue
client_max_body_size
20M;
#proxy_pass_request_headers on;
#proxy_set_header
X-FILE $request_body_file;
#proxy_set_body
off;
#proxy_redirect
off;
#proxy_pass
http://www.example.org/;
sendfile on;
add_header x-peticio $request;
add_header x-temps $request_time;
add_header x-longitud $request_length;
#root html;
#return 200 '$request_body_file';
return 200 'tot bé\n';
}
- Client (POST)
curl -i --data-binary '@myfile.png'
http://192.168.1.29/upload
- Problemes / Problems
- nginx-upload-module
- Upload
- Resumable upload
- nginx-upload-module
2.2 (github)
- Nginx
upload module (v 2.2.0)
- nginx.service
[ Service ]
...
# create hashed dirs for upload-module
(level1=1)
ExecStartPre=/bin/sh -c '/bin/mkdir -p
/var/www/uploads/{0..9}; chmod 777 -R
/var/www/uploads/{0..9}'
- Example configuration
# allow 100 Continue.
# if not specified
here, as general value, defaults to 1M,
and limits upload,
# even if
a greater vallue is specified inside a
location section
# (which only
controls return of 100 Continue message)
client_max_body_size
20M;
location /upload
{
# this value would response 100 for
files under 30M, but general
client_max_body_size 20M
# would not allow upload of files
between 20 and 30M:
#
client_max_body_size
30M;
# Pass altered request body to this
location
upload_pass
@new_upload;
# Store files to this directory
# The directory is hashed:
# level1=1 (digit) : subdirectories 0 1
2 3 4 5 6 7 8 9 should exist
# (they can be created in nginx.service)
upload_store /var/www/uploads 1;
# Allow uploaded files to be read only
by user
upload_store_access user:r;
# Set specified fields in request body
# They are passed to backend as POST
multipart/form-data
upload_set_form_field
"${upload_field_name}_name"
"$upload_file_name";
upload_set_form_field
"${upload_field_name}_content_type"
"$upload_content_type";
upload_set_form_field
"${upload_field_name}_path
"$upload_tmp_path";
# Inform backend about hash and size of
a file
upload_aggregate_form_field
"${upload_field_name}_md5"
"$upload_file_md5";
upload_aggregate_form_field
"${upload_field_name}_size"
"$upload_file_size";
# pass fields from form
# pass all:
#upload_pass_form_field "(.*)";
# pass only submit and description:
upload_pass_form_field
"^submit$|^description$";
upload_cleanup 400 404 499 500-505;
}
# Pass altered
request body to a fastcgi
backend
location @new_upload {
# user-defined headers
add_header x-request-time $request_time;
add_header x-request-length
$request_length;
fastcgi_pass
unix:/var/run/fcgiwrap.socket;
# Fastcgi parameters, include the
standard ones
include /etc/nginx/fastcgi_params;
# Adjust non standard parameters
(SCRIPT_FILENAME)
fastcgi_param SCRIPT_FILENAME
/usr/local/bin/new_upload.sh;
# user-defined parameters:
fastcgi_param REQUEST_LENGTH
$request_length;
fastcgi_param REQUEST_TIME
$request_time;
}
- /usr/local/bin/new_upload.sh
#!/bin/bash
echo -e "Content-type: text/plain\n"
source cgi_functions.sh
# register all GET and POST variables
cgi_getvars BOTH ALL
echo "New upload
==============================="
export
- Other examples
- Compilation
-
|
client
|
response code
|
POST
multipart/form-data |
curl -i
-X POST -H 'Content-Type:
multipart/form-data' -F toto='@myfile.png'
http://<server>/up |
200 OK
|
POST
x-www-form-urlencoded |
curl -i
--data-binary '@myfile.png'
http://<server>/up |
415
Unsupported Media Type |
PUT
|
curl
--upload-file myfile.png http:// <server>/up |
405 Not
Allowed |
- Resources
(.com)
- i18n
- 3rd party modules
- Problemes / Problemes
- check content of:
/var/log/nginx/error.log
/usr/local/nginx/logs/error.log
- ...
- related to fcgiwrap
connect() to unix:/var/run/fcgiwrap.socket
failed (11: Resource temporarily
unavailable) while connecting to
upstream, client: ..., server: ..., request: ...,
upstream:
"fastcgi://unix:/var/run/fcgiwrap.socket:", host:
...
- related to uWSGI (WSGI)
connect() to unix:///var/lib/uwsgi/....sock
failed (11: Resource temporarily
unavailable) while connecting to
upstream
- connection to page gives "502 Bad Gateway"
systemctl status emperor.uwsgi.service
import memcache
ImportError: No module named memcache
- connection to page gives "502 Bad Gateway"
- /var/log/nginx/error.log
connect() to
unix:///etc/uwsgi/wct_backend.sock failed
(13: Permission
denied) while connecting to
upstream ...
- If
it
doesn't work
- Solució / Solution
- regenerate
nginx.pp file
sudo systemctl restart
emperor.uwsgi.service
sudo systemctl restart nginx.service
- connection to page gives "502 Bad Gateway"
- /var/log/nginx/error.log
connect() to
unix:///etc/uwsgi/wct_backend.sock failed
(2: No
such file or directory) while
connecting to upstream ...
- Solució / Solution
- check output from (maybe some
misspelling?):
sudo systemctl status
emperor.uwsgi.service
- check that uwsgi is installed in your
virtualenv
- check uwsgi
configuration
- related to Daphne (ASGI)
connect() failed (111: Connection refused)
while connecting to upstream, client: ..., server:
..., request: ..., upstream: "http://[::1]:9000/ws/...",
host: ...
- IPv6 not activated
- Solució / Solution
- conf.d/my_nginx.conf
upstream
my-upstream {
# use 127.0.0.1
instead of localhost to avoid errors
related to ipv6
server
127.0.0.1:9000;
}
[alert]
2398#0: 1024 worker_connections are not enough
- Optimal
value for Nginx worker_connections
- Solució / Solution
- nginx.conf
events {
# this value must
not exceed:
# - max number of
open files for nginx user and
nginx.service:
# sudo su -
nginx -s /bin/bash -c "ulimit -Hn"
#
grep
LimitNOFILE
/usr/lib/systemd/system/nginx.service
# - total
range of sockets to enable per IP:
sysctl net.ipv4.ip_local_port_range
# default value:
worker_connections 1024;
worker_connections 16384;
}
- set max number of open
files
- set total range of sockets to enable per
IP
sysctl
net.ipv4.ip_local_port_range
sysctl -w
net.ipv4.ip_local_port_range ...
[error] 22590#0: *344 recv() failed (104:
Connection reset by peer) while proxying upgraded
connection, client: ..., server: ..., request:
"GET ...", upstream: "http://127.0.0.1:9000/...",
host: ...
[info] 22590#0: *355 peer closed connection
in SSL handshake while SSL handshaking, client:
..., server: 0.0.0.0:443
accept4()
failed (24: Too many open files)
- Nginx:
24: Too Many Open Files Error And Solution
- nginx 24: Too many open
files
- see also: worker_connections are not enough
- Solució / Solution:
- increase limits for nginx user:
- present values:
sudo su - nginx -s /bin/bash -c
"ulimit -Hn"
sudo su - nginx -s /bin/bash -c
"ulimit -Sn"
- set new values at os level for user nginx:
- /etc/security/limits.d/nginx.conf
# default:
nginx
hard
nofile 4096
nginx
hard
nofile 16384
# default:
nginx
soft
nofile 1024
nginx
soft
nofile 4096
- set new values at service level:
- /usr/lib/systemd/system/nginx.service
systemctl daemon-reload
systemctl restart nginx.service
- check new values in running process:
nginx_pid=$(pgrep -f "nginx:
master process")
cat /proc/${nginx_pid}/limits |
grep -E "Limit|Max open files"
Limit
Soft
Limit
Hard
Limit
Units
Max open
files
16384
16384
files
- perform stress tests
exec: fork failed (12: Cannot allocate memory),
client: ..., server: ...
nginx: [emerg] open() "/etc/nginx/nginx.conf"
failed (13: Permission denied)
- Solució / Solution
sudo setsebool -P
httpd_read_user_content 1
nginx: [emerg] bind() to 0.0.0.0:8000 failed
(13: Permission denied)
nginx: [emerg] bind() to 0.0.0.0:444 failed (13:
Permission denied)
- content is not cached
- Apache
HTTP Server (Apache
Software Foundation)
- Documentation
- How
to
configure Apache2 (ntu)
- Dynamic
Content with CGI (CGI)
- Environment
Variables in Apache
-
|
variable
|
bash
|
PHP
|
Python / Django
|
HttpRequest
|
CONTENT_LENGTH
CONTENT_TYPE
HTTP_ACCEPT_ENCODING
HTTP_ACCEPT_LANGUAGE
HTTP_HOST
HTTP_REFERER
HTTP_USER_AGENT
QUERY_STRING
REMOTE_ADDR
REMOTE_HOST
REMOTE_USER
REQUEST_METHOD
SERVER_NAME
SERVER_PORT
|
|
|
Django
request
and response objects
|
mod_ssl
|
SSL_CLIENT_S_DN
SSL_CLIENT_S_DN
- ...
|
|
|
|
CGI
(nginx fastcgi:
/etc/nginx/fastcgi_params)
|
fastcgi_param
QUERY_STRING
$query_string;
fastcgi_param
REQUEST_METHOD
$request_method;
fastcgi_param
CONTENT_TYPE
$content_type;
fastcgi_param
CONTENT_LENGTH
$content_length;
fastcgi_param
SCRIPT_NAME
$fastcgi_script_name;
fastcgi_param
REQUEST_URI
$request_uri;
fastcgi_param
DOCUMENT_URI
$document_uri;
fastcgi_param
DOCUMENT_ROOT
$document_root;
fastcgi_param
SERVER_PROTOCOL
$server_protocol;
fastcgi_param
REQUEST_SCHEME
$scheme;
fastcgi_param
HTTPS
$https
if_not_empty;
fastcgi_param
GATEWAY_INTERFACE CGI/1.1;
fastcgi_param
SERVER_SOFTWARE
nginx/$nginx_version;
fastcgi_param
REMOTE_ADDR
$remote_addr;
fastcgi_param
REMOTE_PORT
$remote_port;
fastcgi_param
SERVER_ADDR
$server_addr;
fastcgi_param
SERVER_PORT
$server_port;
fastcgi_param
SERVER_NAME
$server_name;
|
|
|
|
user defined
(mod_env)
|
PassEnv ...
SetEnv
VARIABLE toto_value
UnsetEnv ...
|
|
|
|
- Apache
configuration
- MPM - Multi-Processing Modules (2.2)
(2.4)
- Sessions
- Rendiment / Performance
- Apache performance tuning (2.2)
(2.4)
- Càrrega del
servidor / Server load
- mod_status
- Debian / Ubuntu (Apache 2.4)
- /etc/apache2/mods-available/status.conf
<IfModule
mod_status.c>
# Allow server status reports
generated by mod_status,
# with the URL of
http://servername/server-status
# Uncomment and change the
"192.0.2.0/24" to allow access from
other hosts.
<Location /server-status>
SetHandler
server-status
#Require
local
#Require
ip 192.0.2.0/24
#
only from 188.79.*.* :
Require
ip 188.79
</Location>
# Keep track of extended status
information for each request
ExtendedStatus On
# Determine if mod_status displays the
first 63 characters of a request or
# the last 63, assuming the request
itself is greater than 63 chars.
# Default: Off
#SeeRequestTail On
<IfModule mod_proxy.c>
#
Show Proxy LoadBalancer status in
mod_status
ProxyStatus
On
</IfModule>
</IfModule>
sudo a2enmod status
- Old way
- httpd.conf
<IfModule
mod_status.c>
<Location
/server-status>
SetHandler server-status
Order
deny,allow
Deny
from all
allow
from 192.168.1
</Location>
ExtendedStatus On
</IfModule>
- get the status and refresh every 10 seconds:
http://your_ip/server-status ?refresh=10
- MPM
parameters
- parsed
HTML (.shtml)
- TkApache
- Quick reference card
- User
Authentication
- Authentication
(Apache)
- mod_ssl
(https
config)
- Environment variables
SSL_CLIENT_S_DN
SSL_SERVER_S_DN
- ...
- Directives
- mod_auth_certificate
(*)
urpmi apache-mod_auth_certificate
/usr/share/doc/apache-mod_auth_certificate/README
/etc/httpd/conf/httpd.conf:
LoadModule
auth_certificate_module
/usr/lib/apache-extramodules/mod_auth_certificate.so
- Apache::Clean (mod_perl
Developer's
Cookbook)
- Introduction
to
Server Side Includes
- Rewrite
- Ten
Things You Didn't Know Apache (2.2) Could Do
- Apache Mobile
Filter (AMF)
- Modern
Mobile
Redirect Using .htaccess
RewriteCond %{REQUEST_URI} !^/mobile/.*$
RewriteCond %{HTTP_USER_AGENT}
"android|blackberry|ipad|iphone|ipod|iemobile|opera
mobile|palmos|webos|googlebot-mobile" [NC]
RewriteRule ^(.*)$ /mobile/ [L,R=302]
- Detect Mobile
Browser
- Fake http server with netcat
- Cherokee Web Server
(wp)
- Netscape Enterprise
Server
- NCSA HTTPd
- CERN httpd
- Servidors lleugers / Light servers
- Jetty Java HTTP Servlet
Server
- Web
Servers Comparison
- Creating
Net
Sites
- Designing Your
Web Site
- Databases
interfaces
- Utilitats / Utilities
- CGI (Common Gateway Interface)
- CGI on Apache
- FastCGI
- RFC
3875
"The Common Gateway Interface (CGI) Version 1.1"
- The
Web
Developer's Virtual Library course
- CGIHTML
(C routines)
- CGI
Developer's Guide
- Bash
- Bash.CGI
- cgi_functions.sh
#!/bin/bash
# https://oinkzwurgl.org/hacking/bash_cgi/
# Phillippe Kehi <phkehi@gmx.net> and
flipflip industries
# to test this function locally:
# source cgi_functions.sh
# QUERY_STRING="var1=value1" cgi_getvars GET
var1; echo $var1
# QUERY_STRING="var1=value1&var2=value2"
cgi_getvars GET ALL; echo $var1; echo $var2
#
CONTENT_TYPE="application/x-www-form-urlencoded"
... cgi_getvars POST ALL; echo $var1; echo
$var2
# to test this function locally:
# source cgi_functions.sh
# QUERY_STRING="var1=value1" cgi_getvars GET
var1; echo $var1
# QUERY_STRING="var1=value1&var2=value2"
cgi_getvars GET ALL; echo $var1; echo $var2
#
CONTENT_TYPE="application/x-www-form-urlencoded"
... cgi_getvars POST ALL; echo $var1; echo
$var2
# (internal) routine to store POST data
function cgi_get_POST_vars()
{
# check content type
# FIXME: not sure if we
could handle uploads with this..
if [ "${CONTENT_TYPE}" !=
"application/x-www-form-urlencoded" ]
&& ! [[ "${CONTENT_TYPE}" =~
^"multipart/form-data" ]]
then
#
if, in /etc/sysconfig/spawn-fcgi, OPTIONS contains -f,
#
this echo can be seen at
/var/log/nginx/error.log
echo "[`basename $0`] WARNING: received
CONTENT_TYPE is ${CONTENT_TYPE}, but
implemented content types for POST are only:
"\
"application/x-www-form-urlencoded,
multipart/form-data" 1>&2
return
fi
# save POST variables (only
first time this is called)
[ -z "$QUERY_STRING_POST" \
-a
"$REQUEST_METHOD" = "POST" -a ! -z
"$CONTENT_LENGTH" ] && \
read -N $CONTENT_LENGTH RECEIVED_POST
#echo "RECEIVED_POST:
$RECEIVED_POST"
if [[ "${CONTENT_TYPE}" =~
^"multipart/form-data" ]]
then
#
export variables from multipart
boundary=$(echo $CONTENT_TYPE | awk -F=
'{print$2}');
#QUERY_STRING_POST=$(echo "$RECEIVED_POST" |
awk -v b=$boundary 'BEGIN
{RS=b"\r\n";FS="\r\n";ORS="&"} $1 ~
/^Content-Disposition/
{gsub(/Content-Disposition: form-data;
name=/,"",$1); gsub("\"","",$1); print
$1"="$3}')
#
variables must be exported here, because they
can contain ampersands in value
eval $(echo "$RECEIVED_POST" | awk -v
b=$boundary 'BEGIN {RS=b"\r\n";FS="\r\n";ORS="
"} $1 ~ /^Content-Disposition/
{gsub(/Content-Disposition: form-data;
name=/,"",$1); gsub("\"","",$1); print "export
"$1"=\""$3"\""}')
else
#
take input string as is
QUERY_STRING_POST="$RECEIVED_POST"
fi
return
}
# (internal) routine to decode urlencoded
strings
function cgi_decodevar()
{
[ $# -ne 1 ] &&
return
local v t h
# replace all + with
whitespace and append %%
t="${1//+/ }%%"
while [ ${#t} -gt 0 -a
"${t}" != "%" ]; do
v="${v}${t%%\%*}" # digest up to the first %
t="${t#*%}"
# remove digested part
#
decode if there is anything to decode and if
not at end of string
if
[ ${#t} -gt 0 -a "${t}" != "%" ]; then
h=${t:0:2}
# save first two chars
t="${t:2}"
# remove these
v="${v}"`echo
-e \\\\x${h}` # convert hex to special char
fi
done
# return decoded string
echo "${v}"
return
}
# routine to get variables from http requests
# usage: cgi_getvars method varname1 [..
varnameN]
# method is either GET or POST or BOTH
# the magic varible name ALL gets everything
function cgi_getvars()
{
[ $# -lt 2 ] &&
return
local q p k v s
# get query
case $1 in
GET)
[ ! -z "${QUERY_STRING}" ] &&
q="${QUERY_STRING}&"
;;
POST)
cgi_get_POST_vars
[ ! -z "${QUERY_STRING_POST}" ] &&
q="${QUERY_STRING_POST}&"
;;
BOTH)
[ ! -z "${QUERY_STRING}" ] &&
q="${QUERY_STRING}&"
cgi_get_POST_vars
[ ! -z "${QUERY_STRING_POST}" ] &&
q="${q}${QUERY_STRING_POST}&"
;;
esac
shift
s=" $* "
# parse the query data
while [ ! -z "$q" ]; do
p="${q%%&*}" # get first part of
query string
k="${p%%=*}" # get the key (variable
name) from it
v="${p#*=}" # get the value from
it
q="${q#"$p"&*}" # strip first part from
query string
#
decode and assign variable if requested
[
-n "$p" ] && [ "$1" = "ALL" -o "${s/
$k /}" != "$s" ] && \
export
"$k"="`cgi_decodevar \"$v\"`"
done
return
}
- CGI
shell script using bash
- Bash
- How
to
parse $QUERY_STRING from a bash CGI script
- Creating
CGI Programs with Bash: Handling POST Data
curl -i -X POST --data 'tata=tata_value'
http://127.0.0.1/test
- Examples with POST
- See also Parse
GET and POST parameters
- simple_cgi.sh
#!/bin/bash -x
echo -e "Content-type: text/plain\n"
echo "content type: ${CONTENT_TYPE}"
echo "QUERY_STRING_POST:"
cat
#!/bin/bash -x
echo -e "Content-type: text/plain\n"
echo "content type: ${CONTENT_TYPE}"
read -N $CONTENT_LENGTH QUERY_STRING_POST
echo "QUERY_STRING_POST:"
echo "$QUERY_STRING_POST"
- parse_multi.sh
#!/bin/bash
echo -e "Content-type: text/plain\n"
echo "CONTENT_TYPE: $CONTENT_TYPE"
echo "CONTENT_LENGTH: $CONTENT_LENGTH"
# get boundary from CONTENT_TYPE
boundary=$(echo $CONTENT_TYPE | awk -F=
'{print$2}')
echo "== boundary:"
echo "$boundary"
# read standard input
read -N $CONTENT_LENGTH QUERY_STRING_POST
echo "== received standard input:"
echo "$QUERY_STRING_POST"
# parse standard input in multipart/form-data
post_params=$(echo "$QUERY_STRING_POST" | awk
-v b=$boundary 'BEGIN
{RS=b"\r\n";FS="\r\n";ORS="&"} $1 ~
/^Content-Disposition/
{gsub(/Content-Disposition: form-data;
name=/,"",$1); gsub("\"","",$1); print
$1"="$3}')
echo "== post_params:"
echo "$post_params"
exit 0
- Perl
- Python
- Servlets / JSP
servers
- C Server Pages
(CSP)
- MIME types configuration
|
Servidors FTP / FTP servers
|
- Clients FTP
- vsftpd
- ProFTPD
- Instal·lació / Installation
- CentOS
sudo yum install proftpd
systemctl start proftpd
- Mini-Howto
- Debugging
- /usr/lib/systemd/system/proftpd.service
ExecStart =
/usr/sbin/proftpd -d10
$PROFTPD_OPTIONS
- Configuring
a directory
- Virtual
users
- Passos / Steps
sudo -i
cd /etc; mkdir proftpd; cd proftpd
wget
https://raw.githubusercontent.com/proftpd/proftpd/master/contrib/ftpasswd
chmod +x ftpasswd
./ftpasswd --group -gid 9000 --name
ftp_users
./ftpasswd --passwd --name user1 --home
/home --shell /bin/bash --uid 8000 --gid 9000
/etc/proftpd.conf
# users
AuthUserFile /etc/proftpd/ftpd.passwd
AuthGroupFile /etc/proftpd/ftpd.group
systemctl restart proftpd.service
- Test
- AWS
and ProFTPD
- security groups
- proftpd.conf
# aws
PassivePorts
1024 1048
MasqueradeAddress
<your_public_ip_address>
- /etc/proftp.conf
DefaultRoot ~
<Directory ~>
<Limit ALL>
DenyAll
</Limit>
<Limit DIRS READ>
AllowAll
</Limit>
</Directory>
<Directory ~/upload>
<Limit WRITE>
AllowAll
</Limit>
</Directory>
|
Clients FTP / FTP Clients
|
- sftp
- lftp
- download
lftpget ftp://user@remote_host/dir1/ dir2/dir3/file1
lftp ftp://user@remote_host/dir1 -e "get
dir2/dir3/file1; bye"
- mirror
lftp ftp://user@remote_host/dir1 -e "mirror;
bye"
- upload
- reverse (-R) mirror
lftp -d -f commands.lftp 2>> toto.log
commands.lftp
open -u
user,password remote_host
set ftp:ssl-allow no
mirror -R
local_dir
- Exclude .git dir:
- lftp
exclude
syntax confusion
commands.lftp
open -u user,password
remote_host
set ftp:ssl-allow no
mirror
--exclude --exclude=^\.git/$ -R
local_dir remote_dir
- How
to
use rsync over ftp
- curlftpfs (mount locally)
- weex (updating web
pages)
- Python
|
|
- Pàgines
web
dinàmiques / Dynamic web pages
- PEAR packages
- PHP CLI (command line)
- PHP CLI
- Using
PHP from the command line
- Instal·lació / Installation
- Exemples / Examples: (-r to avoid
<?php...?> )
- php [-f] toto.php
php -r "mail('to_user@example.org','subjecte','cos
del
missatge','From:from_user@example.org' );"
- IDE
- PHP debug
php -i
/etc/php.ini
(Mageia: urpmi
php-ini ) or /etc/php5/apache2/php.ini
(Debian/Ubuntu)
service httpd restart (or: sudo
service apache2 restart )
- comprova la sintaxi / check syntax
urpmi php-cli
php -l toto.php
/etc/php.ini or /etc/php5/apache2/php.ini
- logs
- error_log
- Examples
error_log("You messed up!", 3,
"/var/tmp/my-errors.log");
error_log(print_r($variable,
TRUE));
error_log("You messed up!", 3,
"php://stdout");
- see also Apache
logs
- check the output
php -f toto.php
- if result is empty and "
echo $? " returns
255, it could mean that dependencies not satisfied
- xdebug
- Instal·lació / Installation
- Mageia
- Ubuntu
sudo apt-get install php5-xdebug
- Eclipse
- Debugging
using XDebug
- Setup php ini file:
- Mageia:
/etc/php.d/A29_xdebug.ini
- Ubuntu:
/etc/php5/apache2/conf.d/20-xdebug.ini
xdebug.remote_enable =
1
xdebug.remote_handler = 'dbgp'
- An http server must be configured and running
- Eclipse: Window / Preferences / PHP / Servers
- Server
- Base URL: http://localhost
- Document Root: /var/www/html
- Debugger: XDebug
- Problems
- Progress tab: "waiting for xdebug session"
- Solution
xdebug.remote_handler = 'dbgp'
- kdevelop
- Debugging
remote
CLI with phpstorm
- Dependencies
- Errors
- Reference - What does this
error mean in PHP?
"Warning: cannot modify header information..."
- Si a php.ini hi ha:
error_reporting = E_ALL & ~E_DEPRECATED
- i es produeix un "Notice:...", es posarà al text
de resposta, i el codi serà un 200, i ja no es
podrà canviar per un altre
- En canvi, si a php.ini hi ha:
error_reporting = E_ALL & ~E_DEPRECATED
& ~E_NOTICE
- no s'escriurà cap "Notice:..." a la resposta, i el
codi HTTP es podrà canviar mitjançant un
"header("HTTP/1.0 204 No Response")"
- Penseu també a eliminar les línies en blanc o
retorns de carro fora de <php>
- i18n
- gettext
- PHP
internationalization
with gettext tutorial
- Localizing
PHP
Applications “The Right Way”, Part 1
- PHP
internationalization
with gettext tutorial
- i18
for
WordPress Developers
- Accept-Language (HTTP
request header)
- dependencies
- Mageia
urpmi php-gettext php-intl
- Steps
cd public
- create a php file for testing:
- public/test_locale.php
<?php
// get language from browser preferences
(Accept-Language http header)
// only works for the first option in the
browser
$lang_from_http_header =
Locale::acceptFromHttp($_SERVER['HTTP_ACCEPT_LANGUAGE']);
echo "lang_from_http_header: " .
$lang_from_http_header . "<br/>";
putenv("LANGUAGE=".$lang_from_http_header);
// setup directory and domain (file name:
<domain>.mo)
$domain = "messages";
bindtextdomain($domain,"Locale");
bind_textdomain_codeset($domain, 'UTF-8');
textdomain($domain);
// why is this needed?
// this locale has nothing to do with the
language
// this locale must be installed in the
system
$locale = "en_US";
setlocale(LC_MESSAGES, $locale);
echo _("Hello world!");
//phpinfo();
?>
- extract all translatable strings to a portable
object template file:
xgettext --from-code=UTF-8 -o php.pot
*.php
find . -iname "*.php" -exec xgettext
-a
-L PHP --from-code=UTF-8 -j -o php.pot {} \;
- create dir structure
mkdir -p Locale/ca/LC_MESSAGES
mkdir -p Locale/fr/LC_MESSAGES
...
- edit messages.pot
"Content-Type: text/plain; charset=UTF-8\n"
- create po files from pot
cp php.pot
Locale/ca/LC_MESSAGES/messages.po
- cd Locale/ca/LC_MESSAGES
- edit messages.po to update translations
- compile po to mo:
msgfmt -o messages.mo messages.po
- (?) restart Apache server
# systemctl restart httpd
- Problems
- PHP
internationalization
– i18n mechanisms tutorial
- How
to
use Locale::acceptFromHttp without a filter list?
- Smarty
- Smarty
- i18n
- smarty-gettext
- Smarty
gettext
plugin
- Installation
- cd <smarty>/plugins
- wget
https://raw.githubusercontent.com/smarty-gettext/smarty-gettext/master/block.t.php
- wget
https://raw.githubusercontent.com/smarty-gettext/smarty-gettext/master/function.locale.php
- Setup
cd public
- get script to extract translatable strings
wget
https://raw.githubusercontent.com/smarty-gettext/smarty-gettext/master/tsmarty2c.php
chmod +x tsmarty2c.php
- edit your master tpl file:
{locale
path="Locale" domain="messages"}
- edit your_view.tpl
{t}Hello world
from Smarty!{/t}
- extract all translatable strings to a pot
file:
./tsmarty2c.php -o smarty.pot
<path_to>/views/
- merge pot files from *.php and *.tpl
xgettext --from-code=UTF-8 -o
php.pot *.php
msgcat -o messages.pot php.pot
smarty.pot
- create po files from pot
cp messages.pot
Locale/ca/LC_MESSAGES/messages.po
cd Locale/ca/LC_MESSAGES
- edit messages.po to update translations
- compile po to mo:
msgfmt -o messages.mo messages.po
- Next times:
- regenerate pot
xgettext --from-code=UTF-8 -o
php.pot *.php
./tsmarty2c.php -o smarty.pot
<path_to>/views/
msgcat -o messages.pot php.pot
smarty.pot
- update po
msgmerge -U
Locale/ca/LC_MESSAGES/messages.po
messages.pot
msgmerge -U
Locale/fr/LC_MESSAGES/messages.po
messages.pot
...
- recompile mo
msgfmt -o Locale/ca/LC_MESSAGES/ messages.mo
Locale/ca/LC_MESSAGES/ messages.po
msgfmt -o Locale/fr/LC_MESSAGES/ messages.mo
Locale/fr/LC_MESSAGES/ messages.po
...
- Correu electrònic / E-mail
- curl-php
- PHP Curl with port number issue
- POST of array to djangorestframework,
in multipart (PhpMultiPartParser)
<?php
// http://stackoverflow.com/questions/3772096/posting-multidimensional-array-with-php-and-curl
function http_build_query_for_curl( $arrays, &$new
= array(), $prefix = null ) {
if ( is_object( $arrays ) ) {
$arrays =
get_object_vars( $arrays );
}
foreach ( $arrays AS $key =>
$value ) {
$k = isset(
$prefix ) ? $prefix . '[' . $key . ']' : $key;
if (
is_array( $value ) OR is_object( $value ) ) {
http_build_query_for_curl(
$value, $new, $k );
} else {
$new[$k]
= $value;
}
}
}
$ch = curl_init();
$url = "http://127.0.0.1:8000/mymodels/";
$myvalues = array("first", "second");
$post_fields = array("name" => "toto", "myvalues"
=> $myvalues );
http_build_query_for_curl( $post_fields, $post );
print_r($post);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HTTPHEADER,
array('Content-Type: multipart/form-data'));
curl_setopt($ch, CURLOPT_POST, sizeof($post_fields));
curl_setopt($ch, CURLOPT_POSTFIELDS, $post );
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$server_output = curl_exec($ch);
$info = curl_getinfo($ch);
if(curl_errno($ch)) {
echo "something was wrong in your
request";
}
curl_close($ch);
print $server_output;
?>
- Certificats
/
Certificates
- Cross-Origin
Resource
Sharing (CORS)
- Requests
with
credentials
- Server-side
Acess
Control (PHP examples)
- http://foo.example/ (Javascript)
var invocation = new XMLHttpRequest();
var url = 'https://bar.other/'
invocation.onreadystatechange = function() {
if (invocation.readyState!=4) return;
var is_ok = invocation.responseText=='OK';
if (is_ok) {
document.location.href = isok.html
} else {
document.location.href
= isnotok.html
}
}
invocation.open('GET', url, true ); invocation.withCredentials = "true" ;
invocation.send();
- https://bar.other/ (PHP)
<?php
header('Access-Control-Allow-Origin: http://foo.example/');
header('Access-Control-Allow-Credentials: true');
if ($_SERVER['SSL_CLIENT_VERIFY']=='SUCCESS') {
echo 'OK';
} else {
$sslcn = $_SERVER['SSL_CLIENT_S_DN_CN']; $msg = "$sslcn NOT_OK";
echo $msg ;
}
?>
- Nota: si no es fan servir
Access-Control-Allow- (o bé
"Access-Control-Allow-Origin:
*" ),
no estarà disponible responseText
a la URL origen. (Firefox: Cross-domain requests
with credentials return empty)
"Access-Control-Allow-Origin:
*" no pot estar lligat a withCredentials = "true"
- HTML
parsing
- PHP3 (es)
(ch)
- Databases interaction
- Emacs
modes
- Php Mode
(EmacsWiki)
cd /usr/share/emacs/24.2/lisp/progmodes/
wget
http://php-mode.svn.sourceforge.net/svnroot/php-mode/tags/php-mode-1.5.0/php-mode.el
- ~/.emacs
(autoload 'php-mode
"php-mode" "Major mode for editing php code." t)
(add-to-list 'auto-mode-alist '("\\.php$" .
php-mode))
(add-to-list 'auto-mode-alist '("\\.inc$" .
php-mode))
- LBD/EPFL
- Manual (local)
- Tutorial
- PHP Nuke
- Suhosin
(Hardened-PHP project - PHP security)
|
Crawling
|
|
|
- Search engine optimization (SEO) (wp)
|
Robots
|
|
|
|
Servidors de correu / E-mail
servers
|
- Infraestructura /
Infrastructure
- DNS
-
Name
|
Type
|
Value
|
example.org.
|
MX
|
1
mail.example.org
|
example.org.
|
TXT
|
"v=spf1
include:... include:..." |
dddk._domainkey.example.org.
|
TXT
|
"v=DKIM1; k=rsa; p=..." |
- Ports:
- Seguretat / Security
- Seguretat / Security
- Info
- SSL / TLS (fixed ports: 465, 993, 995) -> use STARTTLS instead
- STARTTLS (wp: Opportunistic
TLS)
- passa de no segur a segur / upgrade plain text to
secure
- no calen ports específics / no need for specific
secure ports
- es pot fer servir amb: / can be used with: SMTP,
IMAP, POP3, ...
- DMARC
- SPF - Sender
Policy Framework
- DKIM -
DomainKeys Identified Mail
- Signatura digital de les capçaleres / Digital
signature of headers
- clau privada: ...
- clau pública: DNS TXT record
- About
DKIM (Google)
- DKIMCore
- DNS TXT record:
- Verificació / Verification
- Implementacions / Implementations
|
Clients de correu
electrònic / E-mail clients
|
- Obrir un adjunt winmail.dat
/ Open a winmail.dat attachment:
- Install tnef:
urpmi tnef
apt-get install tnef
- Unpack attachement content:
- Command line
- How
do I test an imap server?
- Testing
POP3
and IMAP servers from the command line in CMD or bash
- imap:
- telnet mail.example.com 143
- ...
- telnet/nc to an SMTP server (after EHLO, responses
starting with 250 show available commands) (cannot
continue
a connection when STARTTLS has been invoked; use openssl
s_client or mailx
instead)
- port 25 (default)
- HELO (basic commands)
$ telnet
mail.example.org 25
Trying x.x.x.x...
Connected to mail.example.org.
Escape character is '^]'.
220 client.example.org ESMTP Postfix
HELO
mail.example.org
250 client.example.org
MAIL
FROM:<localuser@example.org>
250 2.1.0 Ok
RCPT
TO:<remoteuser@example.org>
250 2.1.5 Ok
DATA
354 End data with
<CR><LF>.<CR><LF>
To:<remoteuser@example.org>
From:<localuser@example.org>
Subject:First
test
Hi, you!
.
250 2.0.0 Ok: queued as 489C411EE7AC
QUIT
221 2.0.0 Bye
Connection closed by foreign host.
- EHLO (enhanced commands)
$ telnet
mail.example.org 25
Trying x.x.x.x...
Connected to mail.example.org.
Escape character is '^]'.
220 client.example.org ESMTP Postfix
EHLO
mail.example.org 250-client.example.org
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
...
- port 587 (must be activated; see Postfix
on port 587) and STARTTLS:
$ telnet mail.example.org
587 Trying x.x.x.x...
Connected to mail.example.org.
Escape character is '^]'.
220 client.example.org ESMTP Postfix
EHLO
mail.example.org 250-client.example.org
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
STARTTLS
...
- openssl s_client
openssl s_client -connect mail.example.org:587
-starttls smtp
openssl s_client -starttls smtp -connect
smtp.gmail.com:587 -crlf -ign_eof
- Problemes / Problems
read:errno=0 (and connection is
closed):
- check, on postfix server:
/var/log/maillog
- mail / mailx
- Instal·lació / Installation
- Utilització / Usage
- Linux
and Unix mailx command
- mailx(1)
- Linux man page
- How
can
I use the “mail” command?
- enviament / sending (a local email server, e.g. Postfix,
must be running)
- enviament via un servidor smtp remot (p.ex. Postfix) /
sending though a remote smtp server (p.ex. Postfix)
- Sending
Email
from mailx Command in Linux Using Gmail’s SMTP
mail -s "subject" -S
from=my_user@toto.org -S
smtp=smtp://mail.toto.org
destination_user@example.org
- STARTTLS and authentication
mailx -v -s "$EMAIL_SUBJECT"
-S smtp-use-starttls
-S ssl-verify=ignore
-S smtp-auth=login
-S smtp=smtp://smtp.gmail.com:587
-S
from="$FROM_EMAIL_ADDRESS($FRIENDLY_NAME)"
-S smtp-auth-user=$FROM_EMAIL_ADDRESS
-S
smtp-auth-password=$EMAIL_ACCOUNT_PASSWORD
-S ssl-verify=ignore
-S
nss-config-dir=~/.mozilla/firefox/yyyyyyyy.default/
$TO_EMAIL_ADDRESS
- recepció / reception
- MailX
Tutorial
mail
- next page:
z
- previous page:
z-
- delete:
d<from>-<to>
- list of messages (headers):
h
- ...
- Mozilla
Thunderbird (correu/mail)
- Command
line options
- create a new profile
thunderbird -CreateProfile my_profile
- it will be created at:
~/.thunderbird/xxxxxxxx.my_profile/
- open gui profile manager
- open a profile
thunderbird -p my_profile
- Thunderbird 3
- Redacció d'un missatge en HTML quan per
omissió és en text pla / Compose a message in HTML when
default is plain text:
- Majúscula + click a "Redacta" / Shift + click on
"Compose"
- Expanded
columns
in folder pane
- Signatures
-
Thunderbird
- enllaços
cap
a firefox i altres aplicacions:
~/.thunderbird/.../prefs.js:
- Opening
hyperlinks
from some GTK-based applications doesn't work after
updating Firefox to a new version
-
Bug 58784 - Using
"Make firefox the default web browser" should use
/usr/bin/firefox not
/usr/lib(64)/firefox-<version>/firefox
$ gconf-editor
- /desktop/gnome/url-handlers/http/command
- enabled: no
- com que el valor per omissió és
"enabled=true", això crearà el fitxer
.gconf/desktop/gnome/url-handlers/http/%gconf.xml
(si no, no existeix el fitxer)
- llavors el thunderbird ens preguntarà amb quina
aplicació volem obrir els http: /usr/bin/firefox
- gconf:
~/.gconf/desktop/gnome/url-handlers/http/%gconf.xml
- Al Firefox: Edita / Preferències / Avançat / General /
Comprova-ho ara (predeterminat) (*)
- user_pref("network.protocol-handler.app.http",
"/usr/bin/xdg-open");
- user_pref("network.protocol-handler.app.http",
"/usr/bin/mozilla-firefox");
- user_pref("network.protocol-handler.app.https",
"/usr/bin/mozilla-firefox");
- user_pref("network.protocol-handler.app.ftp",
"/usr/bin/mozilla-firefox");
- user_pref("network.protocol-handler.app.smb",
"/usr/bin/konqueror");
- Locale-Switcher
Extension
- Tips
&
tricks
- Consells
i
trucs (Softcatalà)
- Use
different
Quote Level Colors
- ~/.thunderbird/xxx.default/chrome/userContent.css
/* Quote Levels Colors */
blockquote[type=cite] {
color: navy !important;
background-color: RGB(245,245,245) !important;
}
blockquote[type=cite] blockquote {
color: maroon !important;
background-color: RGB(235,235,235) !important;
}
blockquote[type=cite] blockquote blockquote {
color: green !important;
background-color: RGB(225,225,225) !important;
}
blockquote[type=cite] blockquote blockquote
blockquote {
color: purple !important;
background-color: RGB(215,215,215) !important;
}
blockquote[type=cite] blockquote blockquote
blockquote blockquote {
color: teal !important;
background-color: RGB(205,205,205) !important;
}
- Problem: "This body part will be downloaded on demand"
- View->Display Attachments Inline
- how
to
not include external images on html messages? - Mozilla
- moz-do-not-send true
- En recepció, caldrà acceptar "Mostra el contingut
remot"
- Lectura de fitxers mbox / Read
mbox files
- Reading
an mbox file with Thunderbird
- min_thunderbird.sh
#!/bin/bash
EXPECTED_ARGS=1
if (( $# != $EXPECTED_ARGS ))
then
cat <<EOF
Usage: `basename $0` profile_name
Create a thunderbird profile with given name, and
add a default mail setup, in order to read mbox
files.
EOF
exit 1
fi
# check that thunderbird is installed
if ! command -v thunderbird2 >/dev/null
2>&1
then
echo "ERROR: command
thunderbird does not exist. Please install it
(https://www.thunderbird.net/) before running this
script."
exit 1
fi
profile_name=$1
root_thunderbird_dir=${HOME}/.thunderbird
# create profile
echo "1. creating profile: ${profile_name}"
thunderbird -CreateProfile ${profile_name}
# get profile path
profile_dirname=$(find ${HOME}/.thunderbird -type
d -name "*\.${profile_name}")
echo " created: ${profile_dirname}"
# start thunderbird, to create dirs
echo "2. starting thunderbird to create dirs:
please just cancel and close"
thunderbird -p ${profile_name}
# backup orginal prefs.js
prefs_path=${profile_dirname}/prefs.js
original_prefs_path=${profile_dirname}/prefs.original.js
echo "3. backing up ${prefs_path} ->
${original_prefs_path}"
cp ${prefs_path} ${original_prefs_path}
# modify prefs.js
echo "4. modifying ${prefs_path} to add a minimal
mail config"
cat >>${prefs_path} <<EOF
user_pref("mail.root.none",
"${profile_dirname}/Mail");
user_pref("mail.accountmanager.accounts",
"account1");
user_pref("mail.accountmanager.localfoldersserver",
"server1");
user_pref("mail.account.account1.server",
"server1");
user_pref("mail.server.server1.directory",
"${profile_dirname}/Mail/Local Folders");
user_pref("mail.server.server1.directory-rel",
"[ProfD]Mail/Local Folders");
user_pref("mail.server.server1.hostname", "Local
Folders");
user_pref("mail.server.server1.name", "Carpetes
locals");
user_pref("mail.server.server1.nextFilterTime",
10);
user_pref("mail.server.server1.spamActionTargetAccount",
"mailbox://nobody@Local%20Folders");
user_pref("mail.server.server1.storeContractID",
"@mozilla.org/msgstore/berkeleystore;1");
user_pref("mail.server.server1.type", "none");
user_pref("mail.server.server1.userName",
"nobody");
EOF
# start thunderbird
echo "5. starting thunderbird: just check that
default mail folders have been created and close
it"
thunderbird -p ${profile_name}
# additional mbox files
echo "6. to read mbox files, put them on (or link
them from) ${profile_dirname}/Mail/Local Folders/
and restart thunderbird by using one of the
following:"
echo " thunderbird -p ${profile_name}"
echo " thunderbird -P"
echo "To delete profile ${profile_name}:"
echo " thunderbird -P"
echo "and select Delete Profile..."
echo "IMPORTANT: if you made a symbolic link to
your mbox file from ${profile_dirname}/Mail/Local
Folders/, you can choose Delete Files option.
Otherwise, choose Don't Delete Files."
exit 0
- Gmail
|
|
- List
of
web browsers (wp)
- Browserscope
- Browsers.com
- Herramientas
para una navegación satisfactoria (La Vanguardia)
- "A
virtually
secure browser" (pdf)
- Web
browser standards support
- Automatització / Automation
- User agents
- Plug-ins
- Configuració / Settings
-
Mozilla (*)
- Opera
- WaMCom (signText)
- El Navegador
- Netscape
- NCSA
Mosaic
- Konqueror
- Google
- Microsoft Internet Explorer
- Command line
- curl
- curl apis
- opcions / options:
-
group
|
option
|
description
|
input
|
-X
http_method |
method
|
-H
http_request_header |
request
header (e.g. to specify origin for CORS)
|
output |
-i |
include the HTTP
response header in the output |
-I |
displays only
the HTTP
response
header in the output |
-s |
silent |
-v |
verbose (show
also HTTP
request
headers) |
-O |
write output
to a local file named like the remote file
we get (like wget) |
|
-L |
follow new
location (3xx) |
|
-m,
--max-time <seconds> |
Maximum
time in seconds that you allow
the whole operation to take |
|
--referer
...
|
referer
|
authentication
|
-u
user:password |
|
galetes
/ cookies |
-b filename /
<name>=<value>
|
read
from file / send a single cookie
|
-j
|
|
-c
filename
|
write to file
|
https
|
-k
|
trust
certificate
|
--cacert
ca_certificate
|
specify a
certificte for a trusted CA
|
|
...
|
|
- exemples / examples:
curl -X GET -H
'Accept: text/plain' http://example.com/
curl -X GET -H
'Accept: application/json'
http://example.com/
curl -X GET -u usuari:contrasenya
http://example.com/
- POST data
curl -X POST
--data 'toto=true'
http://example.com/my_api/
curl -X POST
-H 'Content-Type: application/json'
--data-binary '{"toto":true}'
http://example.com/my_api/
curl -X POST
-H 'Content-Type: application/json'
--data-binary '@toto.json'
http://example.com/my_api/
- nested json as multipart (e.g.: to be
able to upload an image at the same time)
curl -X POST -H 'Content-Type:
multipart/form-data' -F name='nom' -F
location.name='Reus' -F
location.point='{"type":
"Point","coordinates":
[2.084205,41.473491]}}' -F
thumbnail=@cover.png http://example.com/my_api/
- GET JSON
curl -H 'Accept: application/json;
indent=4' -u admin:password
http://127.0.0.1:8000/users/
curl -H 'Accept: application/json' -u
admin:password http://127.0.0.1:8000/users/ |
python -m json.tool
curl -X OPTIONS -H 'Accept:
application/json; indent=4' -u admin:password
http://127.0.0.1:8000/users/
curl -I -X GET http://example.com/
- HEAD
- get modification time:
curl --head http://example.org/toto
2>/dev/null | awk
'/Last-Modified/ {$1="";print}'
- condition based on modification time:
- upload a file
curl --upload-file
my_file.png http://...
curl -F filedata=@localfile.jpg
http://example.org/upload
- Authentication
- Django
authentication
# login user1
key=$(curl -X POST -H 'Content-Type:
application/json' --data-binary
'{"username":"user1","password":"mypassword"}'
http://example.org/rest-auth/login/ | awk -F
':' '/key/{gsub(/[}"]/,"",$2); print $2}')
# get user details
curl -X GET -H 'Accept: application/json;
indent=4' -H "Authorization: Token $key" http://example.org /rest-auth/user/
- CORS
- In order to get a response header , you must
specify an Origin:
curl -i -H "Origin:
http://example.com" ...
- How
can
you debug a CORS request with cURL?
- regular CORS
curl -H "Origin:
http://example.com" --verbose ...
- preflight CORS
curl -H "Origin:
http://example.com" \
-H
"Access-Control-Request-Method: POST"
\
-H
"Access-Control-Request-Headers:
X-Requested-With" \
-X OPTIONS
--verbose ...
- referer
curl --referer
"http://example.org/index.html" ...
- download a file honouring Content-Disposition
- Galetes
/ Cookies
- HTTP
Cookies
- start cookie engine and read/write cookies to
a file
- write to file
- read from file
- send a single cookie
- If-Modified-Since
- check if the origin has a resource newer than
a specified date
curl -I -z "Fri, 16 Sep 2016
09:43:09 +0200" ...
curl -I -z
"$(date
-R -d 2016-09-16T07:43:09,328834376+0000)"
...
curl -I -H "If-Modified-Since
$( date
-R -d 2016-09-16T07:43:09 )"
...
- get http response code from bash
- How
to
split the HTTP error code from the contents in
cURL?
- How
to
evaluate http response codes from bash/shell
script?
--write-out
- Example:
# add http_code as
an extra final line, after the response
OUT=$(curl --silent --write-out
'\n%{http_code}' -X GET
http://www.example.org/)
# get the output code of the curl command
RET=$?
if [[ $RET -ne 0 ]]
then
echo "Error $RET"
else
# response is
everything but the last line
response=$(echo "$OUT"
| head -n-1)
echo "Response:
$response"
# http code is the last
line
http_code=$(echo "$OUT"
| tail -n1)
echo "http code:
$http_code"
if [[ $http_code ==
"200" ]]
then
# ok
...
else
# not ok
echo
"response: $response"
exit
$http_code
fi
fi
- several retries (bash function)
#!/bin/bash
function send_curl {
# parameters:
# method
"url" "options"
# output is kept in
global variable curl_response
# return value:
# 0: ok (2xx)
# 1: reached maximum
number of retries
# other https codes
return the sum of digits. e.g.:
# 4: 400
# 7: 403
# 8: 404
# ...
# usage examples:
# send_curl GET
"http://192.168.1.100:8000/path/to/?toto=1&tata=2"
"-H 'Authorization: Token $key'"
# send_curl POST
"http://192.168.1.100:8000/path/to/" "-H
'Authorization: Token $key' -F
tete='titi'"
local method=$1
local url=$2
local options=$3
echo "method: $method"
echo "url: $url"
echo "options:
$options"
local
curl_command="curl --silent --write-out
'\n%{http_code}' -X $method $options
'$url'"
echo "curl_command:
$curl_command"
local return_code=1
local max_retries=10
local retries=0
curl_response=""
while (( retries <
max_retries ))
do
echo "[$retries] curl_commmand:
${curl_command}"
OUT=$(eval "$curl_command")
RET=$?
if [[ $RET -ne 0 ]]
then
echo "Error $RET"
else
http_code=$(echo
"$OUT" | tail -n1)
echo "http code: $http_code"
curl_response=$(echo
"$OUT" | head -n-1)
echo "curl_response: $curl_response"
# 2xx
if [[ $http_code =~ ^"2" ]]
then
echo
"ok"
return_code=0
else
#
return code is the sum of digits of
http_code
return_code=$(expr
$(echo $http_code | sed 's/[0-9]/ +
&/g' | sed 's/^ +//g'))
echo
"not ok: http_code=$http_code ->
return_code=$return_code"
fi
break
fi
sleep 2
(( retries++ ))
done
return $return_code
}
- json paginated result (e.g. from Django rest
framework result)
#!/bin/bash
method=$1
url=$2
options=$3
function get_page {
local method=$1
local url=$2
local options=$3
# execute curl request
curl_command="curl
--silent --write-out '\n%{http_code}' -X
$method $options '$url'"
echo "${curl_command}"
OUT=$(eval
"$curl_command")
RET=$?
# http response code
http_code=$(echo "$OUT"
| tail -n1)
# http response
curl_response=$(echo
"$OUT" | head -n-1)
# total number of
elements, including all pages
curl_count=$(echo
${curl_response} | jq '.count')
# link to next page
curl_next=$(echo
${curl_response} | jq -r 'if .links.next
then .links.next else empty end')
}
function get_all_pages {
local method=$1
local url=$2
local options=$3
# first request
get_page "${method}"
"${url}" "${options}"
retrieved_elements=$(echo ${curl_response}
| jq '.results')
cumulated_json="${retrieved_elements}"
# subsequent requests,
until no next is found
while [[ "${curl_next}"
]]
do
get_page "${method}" "${curl_next}"
"${options}"
retrieved_elements=$(echo ${curl_response}
| jq '.results')
# add new results to cumulated results
cumulated_json=$(echo ${cumulated_json} |
jq ". += ${retrieved_elements}")
done
echo "cumulated json:"
echo
"${cumulated_json}" | jq ''
}
get_all_pages "${method}" "${url}"
"${options}"
exit 0
- mime-type /
content_type
curl_get_command="curl
--silent --write-out '\n%{content_type}\n%{http_code}'
-X GET ..."
OUT=$(eval ${curl_get_command})
RET=$?
echo "OUT: $OUT"
if [[ $RET -ne 0 ]]
then
echo
" Error
$RET"
exit $RET
else
content_type=$(echo
"$OUT" | tail -n2 | head -1)
echo
"
content_type: $content_type"
http_code=$(echo "$OUT"
| tail -n1)
echo
" http_code:
$http_code"
response=$(echo "$OUT"
| head -n-1)
if [[ $http_code ==
"200" ]]
then
echo " ok"
else
echo "
response: $response"
fi
fi
- AWS
user data
# get user-data
# check if there is user_data
http_code=$(curl --write-out '%{http_code}\n'
--head -o /dev/null -s
http://169.254.169.254/latest/user-data/)
if [[ $http_code == "200" ]]
then
user_data=$(curl -s
http://169.254.169.254/latest/user-data/)
...
else
...
fi
- Django
REST
framework from curl
- Django
REST
framework test
- Django
Tastypie
from curl
- HTTPS
- cURL:
Adding/Installing/Trusting
New Self-Signed Certificate
- per a connectar-vos amb un servidor amb un
certificat de servidor emès per una CA / to
connect to a server with a server certificate
issued by a CA:
curl
... --cacert
ca_that_issued_the_server_certificate.crt
... https://...
- per a connectar-vos amb un servidor amb un
certificat autosignat
/ to connect to a server with a selfsigned
server certificate
- aconseguiu el certificat de servidor /
get the server certificate (
autosigned_server_certificate.crt )
curl
... --cacert
autosigned_server_certificate.crt ...
https://...
- o bé afegiu el certificat de la CA (o el
certificat autosignat del servidor) a la llista de
certificats de confiança / or add the CA
certificate (or the selfsigned server certificate)
to the list of trusted certificates:
- trust (selfsigned) certificates:
- Issues with Let's
Encrypt certificates
- GNU
Wget (at
GNU)
-
- galetes
/ cookies
- HTTPS
- client authentication:
wget --certificate=client_pem.crt
--private-key=client_pem.key
--ca-certificate=root_ca_pem.crt https://...
- recursiu / recursive:
-
- Clients FTP
- WGET
software for FTP and Web Auto-mirroring
(alternatives)
- Snarf
- Pavuk
- W3M
- Java-based
|
HTML
|
- Accessibilitat / Accessibility
- Characters, entities
- Desenvolupament /
Development
- General structure of a site
- HTML/CSS Templates
- Editors
- Mag's
Big
List of HTML Editors
- AdvaSoft AsWedit
- ASHE
- tkHTML
(tcl/tk)
- HTML
helper
mode for Emacs
- Quanta
- Blue
Griffon
- Installation
- Compilation
- Problemes /
Problems
- Kompozer
- Kompozer labs
- HTML
Timing
- Compilation
- Kompozer
Dev-Howto
- dependencies
- urpmi gcc-c++ lib64gnome-vfs2-devel
lib64IDL2-devel lib64xt-devel lib64ftgl-devel
lib64freetype2-devel lib64pangox-devel
lib64png12-devel
- svn checkout
https://kompozer.svn.sourceforge.net/svnroot/kompozer/trunk
kompozer
- svn patch this.patch
- kompozer/obj-kompozer/config/autoconf.mk
- XT_LIBS = -lX11 -lXt
FT2_LIBS = -lfreetype -lftgl
- cd kompozer
- cp mozilla/composer/config/mozconfig.fedora
mozilla/.mozconfig
- make -f client.mk build_all
- Problemes
/
Problems (vegeu / see: Bluegriffon)
- Bugs
- <object> makes title disappear from tab
- Nvu
-
- Atom
- gwrite
- Compilation
- dependencies
- urpmi python-distutils-extra intltool
- python-jswebkit
- urpmi python-cython lib64webkitgtk1.0-devel
lib64python-devel
- wget
https://gwrite.googlecode.com/files/python-jswebkit-0.0.3.tar.gz
- tar xvzf python-jswebkit-0.0.3.tar.gz
- cd python-jswebkit-0.0.3
- su; python setup.py install
- gwrite
- wget
https://gwrite.googlecode.com/files/gwrite-0.5.1.tar.gz
- tar xvzf gwrite-0.5.1.tar.gz
- cd gwrite-0.5.1
- su; python setup.py install; ./install
- Wix
(Flash)
- Aptana Studio
3 (Eclipse)
- prerequisites:
- Eclipse: Help -> Install new software -> Add
- Templates:
- Problems:
- Previously installed PyDev must be uninstalled (*)
- Javascript
- Manuals and specifications
- MathML (wp)
- MathML
(MDN - Mozilla)
- W3C Math Home
- Brief
Tutorial on MathML
- Specification
-
|
tokens |
entities |
Presentation MathML |
<mi>x</mi> –
identifiers
<mo>+</mo> –
operators
<mn>2</mn> –
numbers
<mtext>non zero</mtext>
– text
<mrow> – a horizontal
row of items;
<msup>, <munderover> ,
and others – superscripts, limits over and
under operators like sums, etc.;
<mfrac> – fractions;
<msqrt> and
<mroot> – roots;
<mfenced> - surrounding
content with fences, such as parentheses.
|
π
→
⁢
...
|
Content MathML |
|
|
- Lletres / Fonts
- Editors
- HTML5
(draft)(WhatWG
Web
applications 1.0) (wp)
- Estil / Style
- Metadades incrustades / Embedded metadata
- Testers / Validations
- Utilitats / Utilities
|
CSS
|
- Web Style Sheets home page
- CSS (All standards
and drafts) (W3C)
-
- CSS usage in Confluence Export PDF:
- CSS validator
- CSSED - "A GTK-2 CSS
Editor"
- CSS (wp)
- Mozilla
CSS
support chart
-
selector
|
usage
|
example
|
usage from HTML
|
|
for all elements of
specified type
|
h1 {...}
|
<h1>
|
id selector
|
for a single, unique
element
|
#toto {...}
|
<...
id="toto">
|
class
selector
|
for any type of element,
with this class
|
.toto {...} |
<p
class="toto">
<h1 class="toto">
...
|
only for the specified
type, with this class
|
h1.toto {...} |
<h1
class="toto"> |
- Inserció / Insertion
- external
- toto.css
h1
{
/* comment */
color:red;
text-align:center;
}
- html
<head>
<link rel="stylesheet" type="text/css"
href="toto.css" />
</head>
- internal
<head>
<style>
h1
{
color:red;
text-align:center;
}
</style>
</head>
- inline
<body>
<h1 style=" color:red;
text-align:center; "/>
</body>
- Posicionament
/
Positioning
-
|
description
|
original space
preserved
(does it affect the position of other elements?)
(is on the normal flow?)
|
move with scroll
|
static
|
default
|
yes
|
yes
|
fixed
|
relative to browser
window
|
no
|
no
|
relative
|
relative to its
normal position
|
yes
|
yes
|
absolute
|
relative to the first
parent element that has a position other than
static.
If no such element is found, the containing block is
<html>
|
no
|
yes
|
- Margins
- Exemples / Examples
- HbbTV
- mostra el contorn de l'element / Outline:
outline: #00dd00 dotted
medium;
outline-style: dotted;
outline-color: #00dd00;
outline-width: medium;
- canvi de color d'enllaços visitats
a.intern {
color: #bb0000;
}
a.intern:visited {
color: #9b6a5b;
}
- afegir imatge al final
.nou:after
{
content: url(im/nou.png);
}
|
XHTML
|
|
XML
|
- Extensible Markup Language
(XML) (W3C)
- Schemas
- The XML library for Gnome
- <recursos XML/>
- XML Pitstop
- Search
- Validate
- Style
- XSL (EXtensible Stylesheet Language)
- style sheets for XML (CSS = style sheets for HTML)
- Transform
- Formats
- XSLT
-
-
- XML
and XSLT (w3schools)
-
- Conversion XML -> HTML
- browser will take file specified on
xml-stylesheet to render XML as HTML; if no
xml-stylesheet is specified, it will render raw
xml
-
cdcatalog.xml |
cdcatalog.xsl (xml -> html) |
cdcatalog_text.xsl (xml -> txt) |
<?xml version="1.0"
encoding="UTF-8"?>
<?xml-stylesheet
type="text/xsl"
href="cdcatalog.xsl"?>
<catalog>
<cd>
<title>Empire
Burlesque</title>
<artist>Bob
Dylan</artist>
<country>USA</country>
<company>Columbia</company>
<price>10.90</price>
<year>1985</year>
</cd>
.
.
</catalog> |
<?xml version="1.0"
encoding="UTF-8"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output
method="html"/>
<xsl:template
match="/">
<html>
<body>
<h2>My CD
Collection</h2>
<table border="1">
<tr
bgcolor="#9acd32">
<th>Title</th>
<th>Artist</th>
</tr>
<xsl:for-each
select="catalog/cd">
<xsl:sort
select="artist"/>
<tr>
<td><xsl:value-of
select="title"/></td>
<td><xsl:value-of
select="artist"/></td>
</tr>
</xsl:for-each>
</table>
</body>
</html>
</xsl:template>
</xsl:stylesheet> |
<?xml version="1.0"
encoding="UTF-8"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="text"
media-type="text/plain"/>
<xsl:template match="/">
<xsl:for-each
select="catalog/cd">
<xsl:value-of select="title"/>
<xsl:text>: </xsl:text>
<xsl:value-of select="artist"/>
<xsl:text>
</xsl:text>
</xsl:for-each>
</xsl:template>
</xsl:stylesheet> |
- Conversion XML -> JSON
-
- Tools
- Exemples / Examples
- Editors
- XML
Editing (openlaszlo)
- Emacs
- XAE
- modes
- nXML
(*)
- schemas must be in RelaxNG format (.rnc) (conversion from XSD)
- add lines to file
.../nxml-mode-20041004/schema/schemas.xml:
<namespace
ns="urn:mpeg:mpeg4:SAF:2005"
typeId="SAF"/>
<typeId id="SAF"
uri="saf2.rnc"/>
~/.emacs
- (load
".../nxml-mode-20041004/rng-auto.el")
(setq auto-mode-alist
(cons
'("\\.\\(xml\\|xsl\\|rng\\|xhtml\\|xsr\\)\\'"
. nxml-mode)
auto-mode-alist))
- Eclipse
- Help -> Install new software: search for "XML"
- XMLBuddy
- qxmledit
- Conglomerate
- QXmlEditor
- KXML
Editor (KDE)
- MlView
(gnome)
- Commercial
- From bash:
- Biblioteques
/
Libraries
- Compressió / Compression
- Eines / Tools
- Trucs / Tips
- Diferències /
Differences (text
diff)
|
JavaScript
|
- ECMAScript
-
- Documentation
- Tutorials
- Reference
- JavaScript Kit
- JavaScript
Gamelan
Directory
- DevEdge (Netscape)
- Sash
- JavaScript
Calendar
- Free
Cut-and-Paste
JavaScript
- Javascript PC
emulator (micro Linux)
- JSON
- Comparison to XML
- JavaScript Object Notation
JSON
- Search
- JSON
Schema
- Similar to xsd for
xml
- Documents (IETF)
- Aprenentatge / Learning
- Extensió / Extension
- Eines / Tools
- used by:
- OpenAPI
Initiative (OAI) (Swagger)
- "OpenAPI 3.0 uses an extended subset of JSON
Schema Specification Wright Draft 00 (aka Draft
5) to describe the data formats."
- Django Rest Framework: Schemas
- Exemples / Examples
- JSON Schema
Store
- paypal/api-standards/v1/schema/json
(PayPal) (draft-04)
- simple.schema.json
{
"$schema":
"http://json-schema.org/draft-07/schema#",
"definitions": {},
"type": "object",
"properties": {
"name": {
"title": "Name",
"type": "string"
},
"label": {
"title": "Label",
"type": "string",
"enum": ["abc","zxc"],
"options": {
"enum_titles": ["Abc","Zxc"]
}
}
},
"required": [
"name",
"label"
]
}
- JSON-LD
(Linking Data)
- Exemples / Examples
- IPTC
(International Press Telecommunications Council)
- schema.org
- used by:
- Google: Understand
how structured data works
- index.html
<script
type="application/ld+json">
{
"@context":
"https://schema.org",
"@type": "Organization",
"url":
"http://www.example.com",
"name": "Unlimited Ball
Bearings Corp.",
"contactPoint": {
"@type":
"ContactPoint",
"telephone":
"+1-401-555-1212",
"contactType":
"Customer service"
}
}
</script>
- Parsers
- Parsing json with sed and
awk
- How to parse JSON string via command
line on Linux (jq)
- Python
- command line
- jsawk
- resty
(script wrapper for curl)
- jq
- jq play
(interactive player)
- removal
of
nodes, by naming nodes to excise
- Ús / Usage
- Tutorial
- Manual
- Cookbook
jq [options] '<commands>'
file.json
jq -f file.jq ...
- file.jq
<sequential_command_1>
|
<sequential_command_2> |
(<parallel_command_3.1>),
(<parallel_command_3.2>) |
...
cat file.json | jq [options]
'<commands>'
echo $my_var | jq
[options] '<commands>'
-
special
character
|
description
|
|
|
separate
chained commands
|
()
|
group
commands
|
,
|
separate
parallel commands
|
+
|
concatenate
strings
|
- Jq
to replace text directly on file (like sed
-i)
-
- AWS
- mostra el contingut del fitxer / show the
file content
jq '.' toto.json |
sponge toto.json
- mostra el contingut de la variable / show
the variable content
- multiple input files
- show file1
jq -s '.[0]' file1.json
file2.json
- show file2
jq -s '.[1]' file1.json
file2.json
- merge file1 and file2:
- mostra una línia per a cada entrada, amb
dos dels camps separats per un espai:
jq -r '.[] | [.field_1,
.field_2] | join(" ")'
-
-
{
"llista": [
{
"nom": "primer",
"valor": 1,
"preu": 11
},
{
"nom": "segon",
"valor": 2,
"preu": 22
}
]
}
- crea un array amb els valors d'un sol dels
camps:
jq '[.llista[] | .nom]'
toto.json
-
jq '[.llista[] |
.nom] | join(" ")' toto.json
- ordenats:
jq '[.llista[] | .nom] |
sort | join(" ")' toto.json
jq '.llista[] | .nom' toto.json
| paste
-sd' '
-
- (?)
my_array=($(echo
$my_json | jq '.llista[] | []'jq
'.llista[] | .nom' toto.json |
paste -sd' '))
my_array=($(echo $my_json |
jq '.llista[] | .nom' | paste -sd'
'))
my_array=($(jq '.llista[] |
.nom' toto.json | paste -sd' '))
- posa en una sola línia la clau i el seu
valor:
jq 'to_entries[] | [.key,
.value] | join(" ")' toto.json
- i es posa en un
#!/bin/bash
input="{
\"el primer\": \"un u\",
\"el segon\": \"un dos\",
\"el tercer\": \"un tres\"
}"
declare -A my_array
while IFS= read -r element
do
echo "-- read
element from jq: $element"
key=$(echo
"$element" | awk 'BEGIN{FS="#"}
{print $1}')
value=$(echo
"$element" | awk 'BEGIN{FS="#"}
{print $2}')
my_array["$key"]="$value"
done < <(echo "$input" | jq
-r 'to_entries[] | [.key, .value]
| join("#")')
for key in "${!my_array[@]}"
do
echo "${key}:
${my_array[$key]}"
done
exit 0
- create json
jq -n '{foo:"bar",foo2:3}'
jq -n '.foo="bar"'
jq -n '.[0].foo=1'
- variables bash
- Afegeix entrades a una llista / Add
entries to a list
part_1=$(jq -n
'[{foo:"bar",foo2:3}]')
part_2=$(jq -n
'[{foo:"bar2",foo2:4}]')
part_1=$(echo $part_1 | jq ". +=
$part_2")
echo $part_1 | jq '.'
- Substitució / Replacement
- Manipulate json as a bash variable:
body=$(jq -n
'{foo:"bar",foo2:3}')
echo "${body}" | jq '.'
body=$(echo "${body}" | jq '. +=
{foo3:"bar3"}')
echo "${body}" | jq '.'
value=bar4
body=$(echo "${body}" | jq ". +=
{foo3:\"${value}\"}")
echo "${body}" | jq '.'
object=$(jq -cr -n
'{foo4:"bar4",foo5:5}')
echo "${object}" | jq '.'
body=$(echo "${body}" | jq ". +=
{foo6:${object}}")
echo "${body}" | jq '.'
- Subconjunt / Subset
jq
'{field1:.field1,field2:.field2}'
- comprova si existeix my_key / check if
my_key exists
my_value=$(cat
toto.json | jq -r 'if .my_key then
.my_key else empty end')
if ! [ "$my_value" ]
then
exit 1
fi
- if "my_key" exists, output "
-my_key
<my_key_value> "
(.my_key |
values | tostring | "-my_key "+.),
- filters
- map
-
".nginx.applications[] | select(.name
==\"$application_name\") | .record"
- Només la primera / Only the first
occurrence:
".nginx.applications[] |
map( select(.name
==\"$application_name\") |
.record) | if .[0] then .[0] else
empty end"
- select elements with
name="my_prefix..." :
'.my_list[] | select(.name |
startswith("my_prefix") )'
- boolean
select(... and ...)
select(... or ...)
- key and value
- Select
objects based on value of variable
in object using jq
- Filter
objects based on tags in an array
- given the following json:
[
{
"title": "first",
"number": 1,
"tags": [
{"key": "foo1","value":
"bar1"},
{"key": "foo2","value":
"bar2"}
]
},
{
"title": "second",
"number": 2,
"tags": [
{"key": "foo1","value":
"bar2"},
{"key": "foo2","value":
"bar1"}
]
},
{
"title": "third",
"number": 3,
"tags": [
{"key": "foo1","value":
"bar1"}
]
}
]
- select elements with a tag
foo1/bar1 (first, third, but not
second):
jq '.[] | select(
.tags[] | . and
.key=="foo1" and
.value=="bar1")'
-
-
- sort
(descending) by sum of several fields
[
.[] | ([.components[] | .bandwidth] |
add ) as $total_bandwidth |
.+{tbw:$total_bandwidth}
] |
sort_by(-.tbw)
- sort and swap first and second elements of
an array
- convert json to ffmpeg parameters
.[] |
[
("-vsync vfr"),
(.components[] |
if .type!="image"
then
(if
.type=="video" then "v" else "a" end)
as $tipus |
#.lang
"-c:"+$tipus, .codec,
(if
.codec=="aac" then "-strict -2" else
empty end),
(.preset | values | if (. | length)
> 0 then "-preset "+. else empty
end),
#(.preset | values | "-"+$tipus+"pre
"+.),
"-b:"+$tipus, (.bandwidth | tostring),
(.channels | values | tostring | "-ac
"+.),
(if
.width then "-vf scale=w="+(.width |
tostring)+":h="+(.height | tostring)
else empty end),
#(if
.width then "-vf scale=w="+(.width |
tostring)+":h="+(.height |
tostring)+":force_original_aspect_ratio=decrease"
else empty end),
#(if
.width then "-vf
scale='-2:ceil(min(1\\,min("+(.width |
tostring)+"/iw\\,"+(.height |
tostring)+"/ih))*ih)'" else empty
end),
(.gop |
values | tostring | "-g "+.),
(.profile | values | if (. | length)
> 0 then "-profile:"+$tipus+" "+.
else empty end),
#(.profile | values | if (. | length)
> 0 then "-profile "+. else empty
end),
(.level
| values | tostring | "-level "+.),
(.keyint_min | values | tostring |
"-keyint_min "+.),
(.sc_threshold | values | tostring |
"-sc_threshold "+.)
else
empty
end
),
(.segment_duration | tostring |
"-hls_time "+.),
(.segment_list_size | tostring
| "-hls_list_size "+.),
(.segment_wrap | tostring |
"-hls_wrap "+.),
(.segment_start_number |
tostring | "-start_number "+.),
.name + ".m3u8"
]
| join(" ")
-
cat toto.json | python -m json.tool
- Documentation
- Conversion
- Typson
(TypeScript -> json-schema)
- Debugging
Javascript
- JSONP (wp)
- Toolkits, Frameworks
- Comparison
of
Javascript frameworks
- CAAT
(multi-instance director-based scene-graph manager)
(animacions 2D)
- APE project
(Ajax Push Engine)
- mootools
- Node.js
-
|
name
|
info
|
installation
|
usage
|
Node.js |
|
Installation of
Node.js on system
|
System:
- Install
NodeJS (Angular JS 1: Tutorial)
- Mageia
urpmi nodejs (also
installs npm)
- CentOS
- Debian / Ubuntu
sudo apt-get install
nodejs-legacy
|
|
nvm |
Node.js version
management:
installation of several versions of Node.js on
user home (~/.nvm/ )
|
- no need to install Node.js on system
npm
install -g nvm (NVM
in npm is not the right one? #304)
curl -o-
https://raw.githubusercontent.com/creationix/nvm/v0.35.3/install.sh
| bash
- (logout and login)
|
- see Janus
(WebRTC)
- install and register the latest version of
Node.js
- install and register a specific version of
Node.js
- set an alias for default
- switch to a registered version
- switch to system version
- show current version
- list installed versions
- search for a specific version
|
Package manager
|
npm |
|
- Mageia
urpmi npm (no needed if
installed nodejs)
- CentOS
- Debian / Ubuntu
npm -g install npm@latest
|
- from
package.json , install
locally to node_modules/
- registered package (and install globally:
-g, to
/usr/lib/node_modules/ or ~/.nvm/versions/node/vx.y.z/lib/node_modules/ )
npm install -g angular-cli
[sudo] npm install -g ...
- install locally (no -g) in
node_modules/
and register to package.json (--save):
- list of installed packages
|
Packages
|
angular-cli
|
AngularJS |
npm install
-g angular-cli |
Used by Eclipse
plugin: Angular2
Eclipse
|
Bower |
packages for
deployment manager
- frameworks
- libraries
- assets
- utilities
|
npm install
-g bower |
- from
bower.json , install to
your_project/app/bower_components/
- registered package
|
CoffeeScript
|
little language
that compiles into JavaScript |
npm install
-g coffee-script
|
|
Grunt |
Javascript task
runner |
|
|
http-server
|
HTTP server
|
npm install
-g http-server
|
|
- npm
- AngularJS
(Google)
- ember.js (wp)
- jQuery
(wp)
- qooxdoo (wp)
- Video
players (HTML5)
- Editors
- Packages
- Beautifier
- Exemples / Examples
- Des d'HTML / From HTML
<script type="text/javascript"
src="js/toto.js"></script>
<script type='text/javascript'
language='javascript'>
...
</script>
- background colour
for "a":
- <a onfocus="this.style.background='yellow'"
onblur="this.style.background=''">
- or, in css:
- a:focus {background-color: yellow;}
- redirection
document.location.href = http://...
- location
- Location
object (w3schools)
- Parsing URLs
- location:
http://example.com:8000/toto1/toto2.html#part2
-
|
value
|
location.href |
http://example.com:8000/toto1/toto2.html#part2 |
location.protocol |
http: |
location.host |
example.com:8000 |
location.hostname |
example.com |
location.port |
8000 |
location.pathname |
/toto1/toto2.html#part2
(?)
|
location.hash |
#part2
|
- relative href to a different
port:
- Relative URL to a different
port number in a hyperlink?
- http://192.168.1.10:8080/index.html
// To deal with urls that imply a change of port, on the same server. // They begin with ":". document.addEventListener('click', function(event) { var target = event.target; if (target.tagName.toLowerCase() == 'a') { var parts = target.getAttribute('href').match(/^:(\d+)(.*)/); if (parts) { var adreca_aboluta = location.protocol+"//"+location.hostname+":"+parts[1]+parts[2]; target.setAttribute('href', adreca_absoluta); } } }, false);
<body> ... <a href=":8000/otherpage.html">Other page on the other port at the same server</a> ... </body>
- In this case, before changing the values, they are the
following:
target="http://192.168.1.10:8080/:8000"
target.port="8080"
target.getAttribute('href')=":8000"
- Simple output:
<p>Here is the port:
<script type="text/javascript">
document.write(location.port);
</script>
</p>
- Alert pop-up:
alert("Target port is: "+target.port);
- Debug information:
- Emmagatzematge /
Storage
- Data
/ Date
- Misc
|
|
- Editors
- Eclipse
- typescript-tools
- Installation
npm install -g clausreinke/typescript-tools
- Editors
|
DOM (Document Object Model)
|
|
Dart
|
|
|
|
Gràfics
vectorials
/ Vectorial graphics
|
- SVG
-
- Utilitats /
Utilities (embed into HTML,...)
- W3Schools: SVG
- carto.net
SVG
tutorial, example and demonstration site
- SVG
in
Firefox
- SVG Authoring
Guidelines
- SVG Implementation and
Resource Directory
- Scale-a-vector
- MIME type:
- SVG
1.2
Tiny Test Suite Implementation Matrix
- SVG
animation (wp)
- Unitats / Units
- SVG
Units (w3.org)
- Units
in Inkscape (wiki)
- Understanding
SVG Coordinate Systems and Transformations (Part 1) —
The viewport, viewBox, and preserveAspectRatio
- Definitions
- User Unit
- Unit Identifier
- absolute: mm, cm, in, pt, pc, px
- relative: em, ex, %
- SVG Scale Factor
-
|
desc
|
examples
|
python svgwrite
|
viewport
|
- viewing area; areas outside the viewport are
cropped and not visible
- values are specified as real values (include
a unit identifier; if not specified, assumed
px): 10mm, 2in, ...
|
<svg
width="10in" height="6in" ...>
|
import svgwrite
from svgwrite import in
svg_width = 10
svg_height = 6
dwg = svgwrite.Drawing("toto.svg", profile='full',
size=(svg_width*in, svg_height*in))
|
viewBox
|
- x: from left to right
- y: from top to bottom
- parameters are specified in user units (no
unit identifier)
- SVG scale factor is calculated as:
10in/50user_units = 0.2in/user_unit
|
<svg ... viewBox
= "0 0 50 30"
|
dwg.viewbox(0, 0,
svg_width, svg_height)
|
elements |
- unitless values are considered as specified
in user_units
|
|
|
- Creació / Creation
- Conversió / Conversion
- WMF/EMF (MS Windows)
- Programari
/
Software
-
|
Adobe Flash
|
- Adobe Flash
- Test
- Install
-
navegador
/ browser
|
paquet
del sistema / system package
|
comprovació
de la instal·lació / installation check
|
activació
/ activation
|
notes
|
repo
|
package filename
|
package name
|
provided file
|
Chrome
|
-
|
-
|
-
|
-
|
chrome://components/ |
chrome://settings/content/flash |
Chrome 61: Si
s'activa l'opció «Pregunta-m'ho abans» (que no
preguntarà mai), s'activa la llista blanca
(«Permet») i només aquells llocs que hi estiguin
llistats tindran permís per a executar Flash. |
Firefox
|
Adobe
Flash Player |
flash-player-npapi-27.0.0.130-release.x86_64.rpm
|
flash-plugin -
Adobe Flash Player NPAPI
|
/usr/lib64/mozilla/plugins/libflashplayer.so
->
/usr/lib64/flash-plugin/libflashplayer.so
|
about:addons
|
|
- Programari / Software
- Dispositius
mòbils
/ Mobile devices
|
Altres llenguatges / Other
languages (parsers)
|
|
Desenvolupament / Development
|
|
Serveis web / Web services
|
|
Message-oriented middleware
|
- Info
- Protocols
- Implementations
- Apache
ActiveMQ
- Apache Kafka
(LinkedIn) (wp)
- Apache Qpid (wp)
- RabbitMQ
- Instal·lació / Installation
- Mageia
- Mageia >5, < 5
- Mageia ==5
- Bug
15120 - rabbitmq-server new security
issues fixed upstream in 3.4.1 and 3.4.3
- Install from download:
rpm --import
https://www.rabbitmq.com/rabbitmq-signing-key-public.asc
- valid versions
urpmi
https://www.rabbitmq.com/releases/rabbitmq-server/v3.5.7/rabbitmq-server-3.5.7-1.noarch.rpm
- invalid versions
urpmi
https://www.rabbitmq.com/releases/rabbitmq-server/v3.6.1/rabbitmq-server-3.6.1-1.noarch.rpm
urpmi
https://www.rabbitmq.com/releases/rabbitmq-server/v3.6.0/rabbitmq-server-3.6.0-1.noarch.rpm
- WARNING: rabbitmq 3.6.0-1
requires erlang
"R16B03","5.10.4"; Mageia 5
provides erlang R16B02
- dependencies
urpmi erlang-mnesia
erlang-os_mon erlang-xmerl
erlang-inets erlang-eldap
erlang-public_key erlang-ssl
erlang-tools
erlang-compilererlang-syntax_tools
erlang-crypto
systemctl enable rabbitmq-server.service
systemctl start rabbitmq-server.service
systemctl status rabbitmq-server.service
- Ubuntu
sudo apt-get install rabbitmq-server
- CentOS
- AWS EC2
- Amazon
EC2 (official RabbitMQ site)
- Problemes / Problems
- database is stored in a dir named after
the own ip address, and when the address
changes (using AMI to creae new instances)
the database is lost
- Port
- Accessible from outside (default is ipv6 ::1 only)
- /etc/rabbitmq/rabbitmq.config (IMPORTANT: if
this is the only entry in rabbit section, do not
add comma at the end)
[
{rabbit,
[
{tcp_listeners, [{"0.0.0.0",
5672},
{"::1",
5672}]}
...
- from external computer:
nmap
-p 5672 <rabbitmq_server_ip_address>
- ...
- Check the status
systemctl status rabbitmq-server
rabbitmqctl status
- Problemes / Problems
- check /var/log/rabbitmq/startup_{log, _err}
- E.g.:
{"init terminating in
do_boot",{could_not_start,rabbit,{{erlang_version_too_old,{found,"R16B02","5.10.3"},{required,"R16B03","5.10.4"}},{rabbit,start,[normal,[]]}}}}
rabbitmqctl
...
Error: unable to connect to node
rabbit@localhost: nodedown
- Solució / Solution
rabbitmqctl -n
rabbit@ip-172-31-20-115
-p celery_vhost list_queues
journalctl -u rabbitmq-server /
systemctl status rabbitmq-server.service
Error description:
noproc
- Solució / Solution
- (?) Install erlang (e.g.
sudo
urpmi erlang )
Failed to start RabbitMQ broker
- Solució / Solution
setsebool -P nis_enabled 1
- moreover, if you tried to manually
start rabbitmq server (
sudo
/usr/lib/rabbitmq/bin/rabbitmq-server ),
some files and dirs may have wrong
owner (root instead of rabbitmq):
- journalctl -u rabbitmq-server
{error,{could_not_write_file,"/var/lib/rabbitmq/mnesia/rabbit@ip-xxxx/cluster_nodes.config",...
sudo rm -rf
/var/lib/rabbitmq/mnesia/
sudo systemctl start
rabbitmq-server.service
Event crashed log handler:
(empty /var/log/rabbitmq/rabbit@...log)
- Solució / Solution
- update from erlang-...-R16B-03.16.el7.x86_64
-> erlang-...-R16B-03.18.el7.x86_64
- Configuració / Setup
- Eines / Tools
- rabbitmqctl
- "Only root or rabbitmq should run rabbitmqctl"
rabbitmqctl status
rabbitmqctl list_queues name
messages_ready messages_unacknowledged
rabbitmqctl list_exchanges
- usuaris /
users
rabbitmqctl list_users
rabbitmqctl add_user
<username> <password>
rabbitmqctl add_vhost
<my_vhost>
rabbitmqctl list_vhosts
rabbitmqctl set_permissions -p
<my_vhost> <username> ".*"
".*" ".*"
- set administrator privileges:
rabbitmqctl set_user_tags
<username_administrator>
administrator
- queues
- list
rabbitmqctl [-p my_vhost]
list_queues
- delete / purge (version >=? 3.5.4,
otherwise, use rabbitmqadmin)
- rabbitmqadmin
- you need to activate rabbitmq_management
plugin
- download it from http://localhost:15672/cli/:
curl -O
http://localhost:15672/cli/rabbitmqadmin
chmod +x rabbitmqadmin
- used username has to have administration
permission
- queues
- list
rabbitmqadmin -V celery_vhost -u
<username_administrator> -p
<my_password> list queues
- purge
rabbitmqadmin -V celery_vhost -u
<username_administrator> -p
<my_password> purge queue
name=<queue_name>
- Plugins
- STOMP
- rabbitmq_management
(web / API interface)
/usr/lib/rabbitmq/bin/rabbitmq-plugins
enable rabbitmq_management
- >= v3
- http://localhost:15672
guest / guest
- or any other added user
with administrator privileges
- to see the queues, the user has to
have permission for the specific virtual
host
- to delete a queue, select it and, at
the bottom of the page: Delete / Purge
- API clients
- < v3
- Used by
- Concepts
|
|
exchange
|
publish
|
bind
|
queue
|
|
|
name
|
type
|
routing_key
|
routing_key |
|
Hello World
|
producer
|
''
|
|
'hello' |
|
'hello'
|
|
consumer
|
|
|
|
|
'hello' |
Work queues |
producer |
''
|
|
'task_queue' |
|
'task_queue' |
|
consumer |
|
|
|
|
'task_queue' |
Publish /
Subscribe
|
producer |
'logs'
|
'fanout'
|
|
|
|
|
consumer |
'logs' |
'fanout' |
|
-
|
'amq.gen-xxxx' |
Routing
|
producer |
'direct_logs'
|
'direct'
|
severity |
|
|
|
consumer |
'direct_logs' |
'direct' |
|
severity
|
'amq.gen-xxxx' |
Topics
|
producer |
'topic_logs'
|
'topic'
|
x.y.z
|
|
|
|
consumer |
'topic_logs' |
'topic' |
|
x.y.z
|
'amq.gen-xxxx' |
RPC
|
producer |
|
|
|
|
|
|
consumer |
|
|
|
|
|
- Tutorials (rabbitmq-tutorials)
|
concepts
|
code
|
|
|
producer
|
consumer
|
Hello
World
|
|
#!/usr/bin/env
python
import pika
connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
channel = connection.channel()
|
#!/usr/bin/env
python
import pika
connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
channel = connection.channel()
|
channel.queue_declare(queue='hello') |
channel.queue_declare(queue='hello') |
channel.basic_publish(exchange='',
routing_key='hello',
body='Hello
World!')
print " [x] Sent 'Hello World!'" |
print ' [*]
Waiting for messages. To exit press CTRL+C'
def callback(ch,
method, properties, body):
print " [x] Received %r" %
(body,)
channel.basic_consume(callback,
queue='hello',
no_ack=True)
channel.start_consuming() |
connection.close() |
|
Work
queues
|
- round-robin dispatching
- acknowledgements
- durability
|
#!/usr/bin/env
python
import pika
import sys
connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
channel = connection.channel()
|
#!/usr/bin/env
python
import pika
import time
connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
channel = connection.channel()
|
channel.queue_declare(queue='task_queue',
durable=True) |
channel.queue_declare(queue='task_queue',
durable=True) |
message = '
'.join(sys.argv[1:]) or "Hello World!"
channel.basic_publish(exchange='',
routing_key='task_queue',
body=message,
properties=pika.BasicProperties(
delivery_mode =
2, # make message persistent
))
print " [x] Sent %r" % (message,) |
print ' [*]
Waiting for messages. To exit press CTRL+C'
def callback(ch,
method, properties, body):
print " [x] Received %r" %
(body,)
time.sleep( body.count('.') )
print " [x] Done"
ch.basic_ack(delivery_tag
=
method.delivery_tag)
channel.basic_qos(prefetch_count=1)
channel.basic_consume(callback,
queue='task_queue')
channel.start_consuming() |
connection.close() |
|
Publish
/
Subscribe
|
- exchange
- direct
- topic
- header
- fanout
- temporary queues
- binding
|
#!/usr/bin/env
python
import pika
import sys
connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
channel = connection.channel()
|
#!/usr/bin/env
python
import pika
connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
channel = connection.channel()
|
channel.exchange_declare(exchange='logs',
type='fanout') |
channel.exchange_declare(exchange='logs',
type='fanout') |
|
result = channel.queue_declare(exclusive=True)
queue_name = result.method.queue
|
|
channel.queue_bind(exchange='logs',
queue=queue_name) |
message = '
'.join(sys.argv[1:]) or "info: Hello World!"
channel.basic_publish(exchange='logs',
routing_key='',
body=message)
print " [x] Sent %r" % (message,) |
print ' [*]
Waiting for logs. To exit press CTRL+C'
def callback(ch,
method, properties, body):
print " [x] %r" % (body,)
channel.basic_consume(callback,
queue=queue_name,
no_ack=True)
channel.start_consuming() |
connection.close() |
|
Routing
|
|
#!/usr/bin/env
python
import pika
import sys
connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
channel = connection.channel()
|
#!/usr/bin/env
python
import pika
import sys
connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
channel = connection.channel()
|
channel.exchange_declare(exchange='direct_logs',
type='direct') |
channel.exchange_declare(exchange='direct_logs',
type='direct') |
|
result = channel.queue_declare(exclusive=True)
queue_name = result.method.queue |
|
severities =
sys.argv[1:]
if not severities:
print >> sys.stderr,
"Usage: %s [info] [warning] [error]" % \
(sys.argv[0],)
sys.exit(1)
for severity in severities:
channel.queue_bind(exchange='direct_logs',
queue=queue_name,
routing_key=severity)
|
severity =
sys.argv[1] if len(sys.argv) > 1 else 'info'
message = ' '.join(sys.argv[2:]) or 'Hello
World!'
channel.basic_publish(exchange='direct_logs',
routing_key=severity,
body=message)
print " [x] Sent %r:%r" % (severity, message) |
print ' [*]
Waiting for logs. To exit press CTRL+C'
def callback(ch, method, properties, body):
print " [x] %r:%r" %
(method.routing_key, body,)
channel.basic_consume(callback,
queue=queue_name,
no_ack=True)
channel.start_consuming() |
connection.close() |
|
Topics
|
|
#!/usr/bin/env
python
import pika
import sys
connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
channel = connection.channel()
|
#!/usr/bin/env
python
import pika
import sys
connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
channel = connection.channel()
|
channel.exchange_declare(exchange='topic_logs',
type='topic') |
channel.exchange_declare(exchange='topic_logs',
type='topic') |
|
result = channel.queue_declare(exclusive=True)
queue_name = result.method.queue
|
|
binding_keys =
sys.argv[1:]
if not binding_keys:
print >> sys.stderr,
"Usage: %s [binding_key]..." % (sys.argv[0],)
sys.exit(1)
for binding_key in binding_keys:
channel.queue_bind(exchange='topic_logs',
queue=queue_name,
routing_key=binding_key)
|
routing_key =
sys.argv[1] if len(sys.argv) > 1 else
'anonymous.info'
message = ' '.join(sys.argv[2:]) or 'Hello
World!'
channel.basic_publish(exchange='topic_logs',
routing_key=routing_key,
body=message)
print " [x] Sent %r:%r" % (routing_key, message) |
print ' [*]
Waiting for logs. To exit press CTRL+C'
def callback(ch, method, properties, body):
print " [x] %r:%r" %
(method.routing_key, body,)
channel.basic_consume(callback,
queue=queue_name,
no_ack=True)
channel.start_consuming() |
connection.close() |
|
RPC
|
- callback queue
- correlation_id
- reply_to
|
#!/usr/bin/env
python
import pika
import uuid
class FibonacciRpcClient(object):
def __init__(self):
self.connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
self.channel = self.connection.channel()
result = self.channel.queue_declare(exclusive=True)
self.callback_queue = result.method.queue
self.channel.basic_consume(self.on_response,
no_ack=True,
queue=self.callback_queue)
def on_response(self,
ch,
method, props, body):
if
self.corr_id == props.correlation_id:
self.response
= body
def call(self,
n):
self.response = None
self.corr_id = str(uuid.uuid4())
self.channel.basic_publish(exchange='',
routing_key='rpc_queue',
properties=pika.BasicProperties(
reply_to
= self.callback_queue,
correlation_id
= self.corr_id,
),
body=str(n))
while
self.response is None:
self.connection.process_data_events()
return int(self.response)
fibonacci_rpc = FibonacciRpcClient()
print " [x] Requesting fib(30)"
response = fibonacci_rpc.call(30)
print " [.] Got %r" % (response,)
|
#!/usr/bin/env
python
import pika
connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
channel = connection.channel()
channel.queue_declare(queue='rpc_queue')
def fib(n):
if n == 0:
return 0
elif n == 1:
return 1
else:
return fib(n-1) + fib(n-2)
def on_request(ch, method, props, body):
n = int(body)
print " [.] fib(%s)" %
(n,)
response = fib(n)
ch.basic_publish(exchange='',
routing_key=props.reply_to,
properties=pika.BasicProperties(correlation_id
= \
props.correlation_id),
body=str(response))
ch.basic_ack(delivery_tag =
method.delivery_tag)
channel.basic_qos(prefetch_count=1)
channel.basic_consume(on_request,
queue='rpc_queue')
print " [x] Awaiting RPC requests"
channel.start_consuming()
|
- Mosquitto
- Info
- Instal·lació / Installation
- Ús / Usage
- servidor / server
systemctl enable mosquitto.service
systemctl start mosquitto.service
- client
- producer
mosquitto_pub -h <broker> -t
<topic> -m <message>
- rtl_433 -F
"mqtt://localhost:1883"
- consumer
mosquitto_sub -h <broker> -t
<topic>
- subscribe to all topics
- ...
- ...
|
http://www.francescpinyol.cat/www.html
Darrera modificació: 23 de desembre de 2020 / Last update: 23rd
December 2020
Cap a casa / Back home. |