add_mime nows allocate dinamically copies of the passed strings, so
that we can actually free what we parse from the config file.
This matters a lot especially with lengthy `types' block: strings that
reach the internal mapping are never free'd, so every manual addition
is leaked.
Long time ago, client->req was a static buffer so the memcpy was safe.
However, it's been since moved to a dynamically allocated string, so
it's very often smaller than sizeof(req.buf) (1024), hence the out of
bound access which results in a SIGSEGV very often on OpenBSD thanks to
Otto' malloc.
The situation with the iri parser, client->req and how the request is
forwarded to the other process needs to be improved: this is just a fix
to address the issue quickly, a better one would be to restructure the
iri parser APIs and rethink how the info is forwarded to the ex process.
i.e. allow client_by_id to fail and return NULL.
Initially I thought it was a good idea to shut down a server process
if we receive an invalid client id as reply from one of our requests
to the executor process. This turned out not to be correct since a
client can (read: will) disconnect in the delay beteewn we acknowledge
their request and the cgi script execution.
The fastcgi and proxy handler already handled this situation, so
they're unaffected.
This allows an attacker to make gmid unresponsible by just making
enough requests until they hit the right timing.
it doesn't make any sense to keep the proxying info per-location:
proxying only one per-vhost. It can't work differently, it doesn't make
sense anyway.
Add to gmid the ability to forwad a request to another gemini server and
thus acting like a reverse proxy. The current syntax for the config
file is
server "example.com" {
...
proxy relay-to host:port
}
Further options (like the use of custom certificates) are planned.
cf. github issue #7
Currently dogfooding this patch at gemini.sgregoratto.me. To test,
run the following command and look for the "OCSP response" header:
openssl s_client -connect "gemini.sgregoratto.me:1965" -status
This adds a barebone dumping of the parsed configuration. It is not
complete, but I'm interested in dumping the full path to `cert' and
`key' in order to write some scripts that can inspect the
configuration, extract the certificates and renew them when expired
automatically.
It's not easy to parse gmid configuration otherwise because the syntax
is flexible and users can use macros. Instead, the idea is to run
gmid and let it dump the configuration once it's been parsed in a
static and predictable format.
Now is possible to parse gmid configuration with, say, awk or perl.
From day one we've been using a static array of client struct to hold
the clients data. This has variuos drawbacks, among which:
* reuse of the storage ("shades of heartbleed")
* maximum fixed amount of clients connected at the same time
* bugs are harder to debug
The last point in particular is important because if we mess the client
ids, or try to execute some functions (e.g. the various fcgi_*) after a
client has been disconnected, it's harder to "see" this "use after
free"-tier kind of bug.
Now I'm using a splay tree to hold the data about the live connections.
Each client' data is managed by malloc. If we try to access a client
data after the disconnection we'll probably crash with a SIGSEGV and
find the bug is more easy.
Performance-wise the connection phase should be faster since we don't
have to loop anymore to find an empty spot in the clients array, but
some operations could be slightly slower (compare the O(1) access in an
array with a SPLAY_FIND operation -- still be faster than O(n) thought.)
FastCGI is designed to multiplex requests over a single connection, so
ideally the server can open only one connection per worker to the
FastCGI application and that's that.
Doing this kind of multiplexing makes the code harder to follow and
easier to break/leak etc on the gmid side however. OpenBSD' httpd
seems to open one connection per client, so why can't we too?
One connection per request is still way better (lighter) than using
CGI, and we can avoid all the pitfalls of the multiplexing (keeping
track of "live ids", properly shut down etc...)
This is a big change in how gmid handles I/O. Initially we used a
hand-written loop over poll(2), that then was evolved into something
powered by libevent basic API. This meant that there were a lot of
small "asynchronous" function that did one step, eventually scheduling
the re-execution, that called each others in a chain.
The new implementation revolves completely around libevent'
bufferevents. It's more clear, as everything is implemented around the
client_read and client_write functions.
There is still space for improvements, like adding timeouts for one, but
it's solid enough to be committed as is and then further improved.
This changes the fastcgi implementation from a blocking I/O to an
async implementation on top of libevent' bufferevents.
Should improve the responsiveness of gmid especially when using remote
fastcgi applications.
we need to delete the events associated with the backends, otherwise
the server process won't ever quit.
Here, we add a pending counter to every backend and shut down
immediately if they aren't handling any client; otherwise we try to
close them as soon as possible (i.e. when they close the connection to
the last connected client.)
Macros can be defined at the top of the configuration file:
dir = "/var/gemini"
cert = "/etc/keys"
and re-used later, for example
server "foo" {
root "$dir/foo" # -> /var/gemini/foo
cert "$cert/foo.pem" # -> /etc/keys/foo.pem
}
GMID_VERSION follows the CGI/FastCGI style, i.e. project_name/version.
Define GMID_STRING with a more "human" variant "project_name version",
and reuse that in the --help and --version codepath.
The actual implementation is based off doas' parse.y. This gave us
various benefits, like cleaner code, \ to break long lines, better
handling of quotes etc...
the logger process now can receive a file descriptor to write logs
to. At the moment the logic is simple, if it receives a file it logs
there, otherwise it logs to syslog. This will allow to log on custom
log files.
Not production-ready yet, but it's a start.
This adds a third ``backend'' for gmid: until now there it served
local files or CGI scripts, now FastCGI applications too.
FastCGI is meant to be an improvement over CGI: instead of exec'ing a
script for every request, it allows to open a single connection to an
``application'' and send the requests/receive the responses over that
socket using a simple binary protocol.
At the moment gmid supports three different methods of opening a
fastcgi connection:
- local unix sockets, with: fastcgi "/path/to/sock"
- network sockets, with: fastcgi tcp "host" [port]
port defaults to 9000 and can be either a string or a number
- subprocess, with: fastcgi spawn "/path/to/program"
the fastcgi protocol is done over the executed program stdin
of these, the last is only for testing and may be removed in the
future.
P.S.: the fastcgi rule is per-location of course :)
this fixes a bug introduced with the prefork mechanics: every server
process shared the same socket, and this would cause a race condition
when multiple server processes asked for a script cgi being executed.
This gives each server process its own socket to talk to the executor,
so the race cannot happen.
Include gmid.h as first header in every file, as it then includes
config.h (that defines _GNU_SOURCE for instance).
Fix also a warning about unsigned vs signed const char pointers in
openssl.
Using BUFSIZ in sbuf is not OK. It's variable, and in various places
we assume that sbuf is 1024 (like handle_cgi_reply). We could patch
those, but we aren't sure BUFSIZ is >= 1024! Let's keep the hardcoded
number.
(found by debugging on arch on amd64, where BUFSIZ is bigger)
Before we mmap(2) file for reading, and use a buffer to handle CGI
scripts. Turns out, for sequential access over the whole mmap isn't
better than our loop on read. This has also the additional advantage
that we can use handle_cgi (now handle_copy) for both files and CGI,
which is pretty cool.
This also fixes a nasty bug where we could hang a connection forever,
because we scheduled the wrong type of event (read on POLLOUT and
write on POLLIN, it's the other way around!)
keep mark_nonblock in utils.c, as otherwise the build for the regress
suite will fail (mark_nonblock needs fatal which is in gmid.c, and
we can't link gmid.o with the regress suite...)
Now that I got rid of the enum+switch, adding more state is easier.
Before, we used an hack to remember if we had read the CGI reply or
not (c->code = -1).
This introduces a new state, handle_cgi_reply that reads the CGI
script reply, logs it, and only then switches to handle_cgi.
handle_cgi itself is cleaner, now it only reads into c->sbuf and send
what it had red.
We even get, almost for free, the 42 error. If read exists with -1 or
0 from in handle_cgi_reply, we return a proper error to the client.
We can extend this further in the future and also try to validate the
CGI reply (for now we're only looking for a \n).
instead of having a flag to discern between two different behaviours
in S_SENDING, split that state into S_SENDING_FILE and S_SENDING_CGI
(this will also make it easier in the future to add other sending
states). While there, also get rid of `goodbye' and make start_reply
advance the state machine by itself.
Omar Polo
Stephen Gregoratto