||[Jan. 24th, 2007|02:00 pm]
Mark pointed out that I haven't mentioned anywhere public parts of the security features new in FC6. Well, here it is.
One of the remaining attack vectors in the runtime are function pointers in writable memory. Overwrite the value and you can redirect execution. Of course the pointer must actually be used and randomization must be overcome, but it's theoretically possible.
The remedy I've implemented in libc internally is to
encrypt function pointers. I.e., they are not stored as-is but instead in a mangled form. This mangling consists in my code of XOR-ing the pointer value with a random 32/64-bit value. Each process has its own random value. The code was publicly committed back in December 2005 and is in FC6.
The only real challenge was to make this fast. Especially on platforms like x86 which have no fast PC-relative data access. To not use a fixed address the value is stored in the TCB.
What is protected? I hope meanwhile most function pointers in libc. Some are probably still missing and others cannot be handled this way since they are visible to the outside. For some broken programs (including UML) the setjmp change was the biggest. These programs tried to access the stored code address which now is not really useful anymore (program don't know how to decrypt the value). Other pointers which are encrypted are the iconv and atexit structures as well as some function pointer tables people don't really know about, they are completely internal.
Using encryption (instead of canaries) to protect structures like jmp_buf is at least as secure and in addition faster. Question is whether we can extend the use to other parts of the runtime. Runtimes for languages like C++ and Java just scream for such a protection, virtual function tables are a prime target.