Sunday, October 19, 2014

Learn HTTPS by becoming your own CA

Chain of trust

HTTPS depends on a chain of trust, and each step in the chain consists of a certificate that has been signed by its parent certificate. Usually the chain is 2~3 nodes long. The first node in the chain is one of around 50+ Certificate Authorities (CA) certificates configured to be trusted by your computer (and basically all computers world-wide). The last node in the chain is the certificate identifying the domain name of the HTTPS website you are visiting.

Looking at https://www.google.se for example, the chain of trust starts from GeoTrust Global CA, which is already trusted by most computers by default. They then signed Google Internet Authority G2, and specifically allowing Google to sign other certificates on the behalf of GeoTrust. Google then signed a certificate with a Common Name (CN) of *.google.se, which matches the domain name(s) this certificate represents.



Certificate

A certificate is basically a public key, bundled with some key-values (like an expiration-date and, most importantly, a Common Name, CN), and appended with a hash of these things encrypted with the private key that signed the certificate.



In many cases the Common Name in a certificate is the name of a company (like GeoTrust), but for the website certificate (last-node in the chain) the CN must match the domain name (as seen above for *.google.se, which is a wildcard match for www.google.se among others). Wildcard certificates are convenient since they can be reused for any domain names that matches.

Each certificate has a corresponding private and public key.



Certificate sign request (CSR)

Signing a certificate requires two peers (for example: you and GeoTrust) to co-operate. Each of the two peers has their own private key they need to keep secret. You want to send your public key and some key-values (such as the domain name in Common Name (CN)). To do this you create a Certificate sign request (CSR), which is basically a certificate with a missing signature.



When GeoTrust gets your CSR they can do some research to see that you actually own that domain name (usually by sending a verification e-mail to hostmaster@your-domain.com). They can also look up your company in a public directory and/or give you a phone call. When satisfied, they add some key-values such as valid from, valid to and purpose of certificate. (For example, the certificate Google Internet Authority has an extra purpose which gives it right to sign other certificates - but the *.google.se does not.) After adding their signature by (hashing and encrypting that hash with their private key) they send you back a complete certificate.

Let's play Pretend Certificate Authority

First install OpenSSL and Nginx. Put openssl on path and open a terminal to the Nginx conf folder.

1.1) Generate your CA private RSA key (root.key)

openssl genrsa -out root.key 2048

Here the key will be of size 2048 bits (default: 512). And the private key will not be password protected. (Password protection only benefits security if you have a password that can resist trillions of brute-force attempts per second, assuming you don't store it in plain-text in your server config which you probably would do anyway :).

1.2) Generate your CA certificate sign request (root_req.csr)


openssl req -new -key root.key -out root_req.csr

You now need to fill in some info, but the most visible in the Common Name (CN).
I wrote Johan Deckmar as my CA Common Name. After completing all the steps, this is what the browser will show:

(Final result)

1.3) Self-sign the root_req.csr with the root private key to get the CA root certificate (root.crt)

openssl x509 -req -days 365 -in root_req.csr -signkey root.key -out root.crt

Alright. Half way there, almost. Currently the root.key needs to be kept private. And the root.crt is a public certificate that you want people to trust as a root CA certificate. If you are doing this at your company, it's the root.crt that needs to be installed as a root CA cert on the employees computers.

Now, add the root.crt as a trusted CA root cert on your computer.

In Windows: Double-click on it --> Install certificate --> Next --> Select destination: Trusted root certificate authorities --> Next --> Complete.

For managing certs in Windows in general: Start --> Run: certmgr.msc

2.1) Generate your web-server private RSA key (server.key)


openssl genrsa -out server.key 2048

2.2) Generate your web-server certificate sign request (server_req.csr)


openssl req -new -key server.key -out server_req.csr

As before, fill in some info about your company and web-servrer. NOTE: Set the Common Name (CN) to a domain name (or wildcard) matching your webserver (where you have Nginx). If you are just trying this on your local computer, you can use the following DNS which points to your local IP-address:  localhost.deckmar.net

2.3) Sign your web-server CSR with CA private key to make the cert (server.crt)


openssl x509 -req -in server_req.csr -CA root.crt -CAkey root.key -set_serial 02 -out server.crt -days 365

2.4) Concatenate your web-server cert and the root CA cert to form a chain (server.pem)


Soon when your web-server (Nginx) identifies itself with an HTTPS certificate, it should send the whole chain of certificates starting from the web-server cert and finishing with the root CA cert. This might sound complicated, but is actually done by concatinating the certificate files into one file like this:

cat server.crt root.crt > server.pem

The file-ending doesn't actually matter, but I use the file name server.pem for a file with multiple certificates. (PEM is a format which can contain one or more certificates concatenated after each other).

3) Configure and start Nginx


Now open conf/nginx.conf and go down and un-comment the configurations under # HTTPS server.

I made the following three changes in nginx.conf:

ssl_certificate      ../secure/server.pem;
ssl_certificate_key  ../secure/server.key;

ssl_session_cache    none;

The path to the .pem and .key file is relative to the conf folder, so as you can see I put my keys and certs in a new folder called secure in the nginx folder.

Start Nginx. Assuming everything was done correctly you should now get a green lock when you open:
https://localhost.deckmar.net

Since I used mainframe.deckmar.net as the domain name, this is what I see when inspecting the certificate on my server (click the green lock --> certificate information).


In some cases you need to restart your browser completely to use the latest certificate settings on your computer.

Let me know how if goes if you try this out yourself. Especially if you find some mistake in the commands or something missing/wrong in the explanations.

Cheers!

Sunday, August 5, 2012

One-keyboard-button compile your Google Drive-hosted Latex project

Context

First off, you already have your report on Google Drive and have made a compile-script, right? (See earlier post of how to set that up). 

Binding F2 to run the script

Let's say that "C:\thesis\build.bat" is the script that will download and compile your report. If we can bind one keyboard button to run that script (regardless of currently focused application) it could speed things up.

Here's how to beat Windows into running a .bat file by a key-press:

  1. Right-click on "Cmd" in the start-menu and "Send to taskbar" (making a shortcut among the Windows 7 speed-launch-icons)
  2. Shift-right-click the new cmd-icon in the taskbar and press Properties
  3. In "target" box append: /C "C:\thesis\build.bat"
    It should now say %windir%\system32\cmd.exe /C "C:\thesis\\build.bat" in the target-box. 
  4. In the keyboard-shortcut field select with mouse and press (for example) F2.
  5. [Ok]
Now pressing F2 at any time brings up a console windows with your build-script flashing by like a black-and-white Matrix scene, and the disappears. This is great for typing in the Google Drive document and having your pdf open on the other monitor in SumatraPDF which auto-refreshes the PDF when the file is updated.

Thursday, August 2, 2012

Auto-export URL references to Bibtex correctly in Medeley

The problem

Auto-exporting a bibtex file in Mendeley is nice and awsome when writing in Latex, but @misc URL references get their url in a \url tag, not in \howpublished{\url{...}} - which results in the URL not showing up in the References there when compiling the paper.

Solution

In Tools - Options - Document Details - [Web page], check in "Medium" (this is mapped to \howpublished in the exported Bibtex). (Tips: check in Citation Key also to control the \cite key)

Now, in your URL references, write, for example, "Available: http://www.example...." in the "Medium" field in the details pane of your reference. Now the URL shows up correctly in your paper (and is clickable)!


Thursday, March 8, 2012

Write LaTeX in Google Docs, compile locally to pdf

I want to write my MS thesis in LaTeX code, but also from anywhere via Google Docs.

This is how I automatically fetch and compile LaTeX code from Google Docs:

  1. Create the Google Docs document
  2. Put some LaTeX code in it. Hopefully you have some "boilerplate" code you always start with.
  3. (I assume you have a LaTeX compiler installed and know how to write in LaTeX)
  4. Install wget if you don't have it (Windows:  http://gnuwin32.sourceforge.net/packages/wget.htm, Other:  http://ftp.gnu.org/gnu/wget/)
  5. Press Share in your doc and allow anyone with the link to view it:
     
  6. Save. Copy the public link. Change the "/edit" in the end to only "/export?format=txt".
  7. Make a folder on your computer where you will be doing the LaTeX compilation.
  8. Make a script which downloads the doc as raw text file and then compiles it with the latex compiler.

This is how my download-and-compile script looks like:

"path/do/wget" "https://docs.google.com/doc..." -O thesis.tex --no-check-certificate
"path/to/pdflatex.exe" -interaction=nonstopmode "thesis.tex"

Those two lines are basically all it takes. When I run build_thesis.bat it downloads the very latest code and compiles it into a pdf.

Notes
  • Problems with the UTF-8 BOM sneaking in to the beginning? Try using lualatex.exe instead which supports UTF-8.
  • If you want to split up into many .tex files you can make many docs and wget all of them.
  • If you use images it's easiest to simply have them in your working folder.
  • Images in the Google Doc will be ignored in the raw text, so you can have your images in the doc and the code for LaTeX to compile them in for a nice preview.
  • A ToC in the doc follow as text in raw format, so it's best not to use it and instead rely on scrolling.
Tell me your results. Happy LaTeXing!

/Johan

Monday, August 8, 2011

SSH med RSA-nycklar


Översikt

För att börja använda RSA-nycklar (istället för)/(tilsammans med) lösenord för SSH behöver du först göra två saker:
  1. Generera ett nyckel-par med PuTTYgen
  2. Klistra in din publika nyckel på servern i $HOME/.ssh/authorized_keys
  3. Lägg in din privata nyckel i PuTTY och spara anslutningen så du slipper göra om det 

Guide för Windows

Se till att du har både PuTTY och PuTTYgenLadda ner här.
Öppna PuTTYgen och generera ett nyckel-par med som är minst 1024 bitar.

Lägg sedan ditt namn i kommentaren, t.ex "rsa-key-John", och (om du vill) ett lösenord som måste skrivas för att nyckeln ska kunna användas.
Varför ha ett lösenord på nyckeln?


Om en hacker får tag på din privata nyckel måste han/hon först brute-force:a RSA-nyckelns lösenord för att kunna logga in på server. Det ger tid för en administratör att upptäcka läckan och blockera den/de läckta nycklarnas tillgång till servern.

Spara både den publika och privata nyckeln på din dator. All säkerhet hänger på att din privata nyckel hålls hemlig. Kopiera den text som är markerad i bildan ovanför. Det är din publika nyckel som du ska lägga in på servern.
Hantering av din privata nyckel.


Några riktlinjer:
  • Lägg inte nyckeln i Dropbox
  • Skicka inte nyckeln i e-post
  • Ge inte din nyckel till någon annan - generera istället en ny
  • Använd ett USB-minne om du behöver flytta en privat nyckel
  • Vill du ha en nyckel hemma också? Generera en ny och maila den publika nyckeln så du kan lägga in den från kontoret

Sista steget: Kopiera din publika nyckel som står i rutan "Public key for pasting into..", och klistra in den i $HOME/.ssh/authorized_keys på en egen rad.

Lägg in din privata nyckel i PuTTY och spara anslutningen i Session så du slipper göra om det.



Det anses vara helt säkert att ha en publik SSH-server som endast accepterar RSA-nyckel-inloggning - förutsatt att du håller dina privata nycklar privata. Därför bör du definitivt stänga av lösenords-inloggning i /etc/ssh/sshd_config så att alla brute-force attacker blir meningslösa.

Sunday, November 7, 2010

My opinion on the future structure of the web

With XHTML Strict we have separated content and design. Using CSS we have been able to remove much redundancy in the code delivered to browsers. We basically define snippets of design and apply them on an arbitrary amount of elements.

I believe the next step is to separate data and structure, by defining snippets of structure and applying to on arbitrary amount of data.

This can be easily achieved now using jQuery Templates, which at the time of writing is in beta.

Example:

Imagine the comment-section on YouTube or this blog. Each comment might have an image, a timestamp, username, thumbs-up/down and some wrapping. All this XHTML code is repeated for every comment.

A short comment

Using a short YouTube comment as an example, the XHTML code consists of 721 bytes of data (which has been generated in a loop on the server side and transmitted to the browser). Removing all XHTML code from the example comment - leaving only the actual information needed - yields 48 bytes of data. In other words: The XHTML code contained 6.657% information, the rest being redundant structure code which is repeated for each comment.

Visual representation:

Code of the short comment:

<li data-author-viewing="False" data-id="5mm3YzP7OX3U298MLeHm0HolNSkstTdEdfbrhWTEAnI" data-score="0" data-author="SuperWizTech" data-pending="0" data-blocked="False" data-flagged="False" data-removed="False" class="comment current"> <div class="metadata"> <div> <a class="author" href="/user/SuperWizTech" title="SuperWizTech">SuperWizTech</a> </div> <div> <span class="time">4 months ago</span> </div> </div> <div class="content"> <div class="comment-text" dir="ltr"> <p>completly awsome? :D</p> </div> <div class="metadata-inline"> <a class="author" href="/user/SuperWizTech">SuperWizTech</a> <span class="time">4 months ago</span> </div> </div> </li>

A long comment

The same comparison on a (relatively) long comment yields an information-to-structure-code-ratio of 453 / 1518 = 29.84%

Visual representation

Code of a long comment

<li data-author-viewing="False" data-id="5mm3YzP7OX2-iXk_QNRtPTEnt5jOC_kWffObtTXjW24" data-score="4" data-author="jpsieben7" data-pending="0" data-blocked="False" data-flagged="False" data-removed="False" class="comment current"> <div class="metadata"> <div> <a class="author" href="/user/jpsieben7" title="jpsieben7">jpsieben7</a> </div> <div> <span class="time">2 years ago</span> <span class="comments-rating-positive">4 <img class="master-sprite comments-rating-thumbs-up" src="http://s.ytimg.com/yt/img/pixel-vfl3z5WfW.gif"></span> </div> </div> <div class="content"> <div class="comment-text" dir="ltr"> <p>what you could do is get a rechargeable battery pack and 2 nxts. then have the base hooked up to the wall so it automatically keeps charging and then have the base use the light sensore and send out a strobe which the bartender finds and? goes and gets. a little bit more complicated but would be a bit more acurate and quick. also make the base have a gravity fed drink loader in it so when 1 is taken another takes its place.</p> </div> <div class="metadata-inline"> <a class="author" href="/user/jpsieben7">jpsieben7</a> <span class="time">2 years ago</span> <span class="comments-rating-positive">4 <img class="master-sprite comments-rating-thumbs-up" src="http://s.ytimg.com/yt/img/pixel-vfl3z5WfW.gif"></span> </div> </div> </li>

Summary

Assuming (educated guesses):

  • Average comment text size: 200 bytes (justified for some Unicode encoding)
  • Additional unique data per comment: 50 bytes (such as: index; special class-names)
  • Data structure overhead per comment: 30 bytes (using JSON)

If we were able to represent the data (username, comment-text and timestamp) with said overhead and have its structure defined only once, the comment section on YouTube would only use less than 25% (280 / 1265 = 22.134% in this example) of it's current bandwidth for the comment section for each [page view]/[comment page-flip].

Simply put, separating structure from data on the web would first of all reduce many chunks of redundant code in repetitive sections (such as in comments- and search-result-sections). Secondly the data would - by design - be set free from their client-specific (in this case client = web-browser) delivery.

One benefit from such a separation would be the possibility of easily creating a completely different interaction to the internet service on a mobile client, without the need of making a new API for this.

Sunday, September 13, 2009

"Spotify Friends" Part 2

Part 2. Controlling Spotify playback by injecting keyboard messages

As we disscussed in Part 1, a Windows application has a WndProc that receives messages such as WM_KEYDOWN from the operating system when the user interacts with the window. Now we want to send those messages ourselves to simulate the user pressing shortcuts (such as [space] for play/pause).

We first prepare by spying on the Spotify window with Spy++ (a tool that comes with Microsoft Visual Studio). Attach to Spotify using Spy++ and filter out just the keyboard messages. Now when you press the shortcut keys in Spotify you will see exactly what messages were sent from the OS. This happens when we press [space]:



To send our own [space] message we do the following:
  • Find the Handle of the Spotify window
  • Send the message using PostMessage( .. ) in user32.dll
Finding the Handle (and getting current song) of the Spotify window in C#

We now search for all processes called Spotify. When we find the right process we can get both the handle we are looking for and the title of the main window - which shows what song (if any) is currently playing. If we cannot find the process then we assume Spotify is not running.

spotifyWindow = 0;

Process[] processes = Process.GetProcessesByName("Spotify");
foreach (Process proc in processes)
{
    if (!proc.MainModule.FileName.EndsWith("spotify.exe")) continue;

    if (proc.MainWindowTitle.Length >= 10)
    {
        NowPlaying = proc.MainWindowTitle.Substring(10);
        SpotifyStatus = AppStatus.Playing;
    }
    else
    {
        NowPlaying = "";
        SpotifyStatus = AppStatus.Paused;
    }

    spotifyWindow = (int)proc.MainWindowHandle;
}

if (spotifyWindow == (int)IntPtr.Zero)
    SpotifyStatus = AppStatus.NotRunning;


Sending messages to the Spotify window

Sending [space] press is just a matter of copycatting the messages we saw in Spy++. But first we need to import the PostMessage( .. ) from user32.dll.
[DllImport("USER32.DLL")]

static extern int PostMessage(

    int hwnd,

    int msg,

    int character,

    uint count);



If you are in a WPF project this is a good time to - like I said at the end of Part 1 - add a reference to System.Windows.Forms and add:

using FormsNS = System.Windows.Forms

If you have a Form application you just use Keys directly. To send [space] we do this:
const int WM_KEYDOWN = 0x100;

const int WM_KEYUP = 0x101;

if (spotifyWindow != 0)

{

     PostMessage(spotifyWindow, WM_KEYDOWN, (int)FormsNS.Keys.Space, 1);

     Thread.Sleep(100);

     PostMessage(spotifyWindow, WM_KEYUP, (int)FormsNS.Keys.Space, 1);

}

This time we get away with putting any integer as the last argument in PostMessage, but we will have to be more precise later on when we are sending the more complex shortcuts (like Ctrl+Right).

Sending Ctrl + Key to Spotify

If we look in Spy++ it seems like we can get away with simulating both the [ctrl] and the [right] key at the same time, but that will not work. My theory is that when Spotify gets the [right] key it will look at the keyboard to see if [ctrl] is pressed (because this is easier to do). Our messages so far will not fool Spotify if it looks at the state of the keyboard - so we now need to choose one out of two solutions:
  1. Force ourselves to use [ctrl] + [any other keys] in our global hot-key combo (Spotify will see that [ctrl] is pressed when we send [right] for example)

    or
  2. Use other key combos, and fake that [ctrl] is pressed
If you are satisfied with choice 1 you just need to send the [right] (etc.) keys correctly and you are done. For that reason we will first look at how we send those keys, then look at how we can spoof the [ctrl] key.


Sending [up], [down], [left] or [right]

This time we need to get the last argument in PostMessage correct. We press [ctrl] + [right] in Spotify while spying with Spy++. Then we right-click and look at the Properties on the WM_KEYDOWN message that has VK_RIGHT in it (should be the second one if you cleared the log). In Properties we first see wParam which is the code for the key - (int)FormNS.Keys.Right in this example. We also see the lParam which we need to send as the last argument in PostMessage.

But I have already done that work for you. Here you go:
PostMessage(spotifyWindow, WM_KEYDOWN, (int)FormsNS.Keys.Right, 0x014D0001);
PostMessage(spotifyWindow, WM_KEYUP, (int)FormsNS.Keys.Right, 0xC14D0001);

PostMessage(spotifyWindow, WM_KEYDOWN, (int)FormsNS.Keys.Left, 0x014B0001);
PostMessage(spotifyWindow, WM_KEYUP, (int)FormsNS.Keys.Left, 0xC14B0001);

PostMessage(spotifyWindow, WM_KEYDOWN, (int)FormsNS.Keys.Up, 0x01480001);
PostMessage(spotifyWindow, WM_KEYUP, (int)FormsNS.Keys.Up, 0xC1480001);

PostMessage(spotifyWindow, WM_KEYDOWN, (int)FormsNS.Keys.Down, 0x01500001);
PostMessage(spotifyWindow, WM_KEYUP, (int)FormsNS.Keys.Down, 0xC1500001);



NOTE: Put in a Thread.Sleep(100) between each key you send. I guess the application will not react if the keys are pressed for an extremely short timespan.
If you choose to have [ctrl] in your global hot-keys you now know all you need to know to control Spotify's playback, volume and get the currently playing song.


Faking the [ctrl] key being pressed

To set the [ctrl] key as down or up we need to import the following method in user32.dll:
[DllImport("user32.dll", EntryPoint = "keybd_event", CharSet = CharSet.Auto, ExactSpelling = true)]

static extern void keybd_event(byte vk, byte scan, int flags, int extrainfo);

To press down we pull this one-liner:
keybd_event((byte)FormsNS.Keys.ControlKey, 0x1D, 1, 0);


And to release [ctrl]:
keybd_event((byte)FormsNS.Keys.ControlKey, 0x1D, 1 | 2, 0);

Using these you can now have global hot-keys such as [alt] + [up] (volume up) and [alt] + [right] (next song). When Spotify checks to see the status of the [ctrl] key it will see it as being pressed.

Summary

Looking back we can sum up what we need to do to send key-presses to another window:
  • Get the Handle of the window. The Handle is used by the Windows GUI manager to refer to uniquely identify windows (and other controls).
  • Send the correct messages via PostMessage. Note that the last argument might be important. In that case we can easily get it with Spy++.
  • If we do not want to hold down [ctrl], but we want the application to think [ctrl] is being held down, then we probably need to use keyd_event( .. ) to set [ctrl]