Sunday, August 5, 2012

One-keyboard-button compile your Google Drive-hosted Latex project

Context

First off, you already have your report on Google Drive and have made a compile-script, right? (See earlier post of how to set that up). 

Binding F2 to run the script

Let's say that "C:\thesis\build.bat" is the script that will download and compile your report. If we can bind one keyboard button to run that script (regardless of currently focused application) it could speed things up.

Here's how to beat Windows into running a .bat file by a key-press:

  1. Right-click on "Cmd" in the start-menu and "Send to taskbar" (making a shortcut among the Windows 7 speed-launch-icons)
  2. Shift-right-click the new cmd-icon in the taskbar and press Properties
  3. In "target" box append: /C "C:\thesis\build.bat"
    It should now say %windir%\system32\cmd.exe /C "C:\thesis\\build.bat" in the target-box. 
  4. In the keyboard-shortcut field select with mouse and press (for example) F2.
  5. [Ok]
Now pressing F2 at any time brings up a console windows with your build-script flashing by like a black-and-white Matrix scene, and the disappears. This is great for typing in the Google Drive document and having your pdf open on the other monitor in SumatraPDF which auto-refreshes the PDF when the file is updated.

Thursday, August 2, 2012

Auto-export URL references to Bibtex correctly in Medeley

The problem

Auto-exporting a bibtex file in Mendeley is nice and awsome when writing in Latex, but @misc URL references get their url in a \url tag, not in \howpublished{\url{...}} - which results in the URL not showing up in the References there when compiling the paper.

Solution

In Tools - Options - Document Details - [Web page], check in "Medium" (this is mapped to \howpublished in the exported Bibtex). (Tips: check in Citation Key also to control the \cite key)

Now, in your URL references, write, for example, "Available: http://www.example...." in the "Medium" field in the details pane of your reference. Now the URL shows up correctly in your paper (and is clickable)!


Thursday, March 8, 2012

Write LaTeX in Google Docs, compile locally to pdf

I want to write my MS thesis in LaTeX code, but also from anywhere via Google Docs.

This is how I automatically fetch and compile LaTeX code from Google Docs:

  1. Create the Google Docs document
  2. Put some LaTeX code in it. Hopefully you have some "boilerplate" code you always start with.
  3. (I assume you have a LaTeX compiler installed and know how to write in LaTeX)
  4. Install wget if you don't have it (Windows:  http://gnuwin32.sourceforge.net/packages/wget.htm, Other:  http://ftp.gnu.org/gnu/wget/)
  5. Press Share in your doc and allow anyone with the link to view it:
     
  6. Save. Copy the public link. Change the "/edit" in the end to only "/export?format=txt".
  7. Make a folder on your computer where you will be doing the LaTeX compilation.
  8. Make a script which downloads the doc as raw text file and then compiles it with the latex compiler.

This is how my download-and-compile script looks like:

"path/do/wget" "https://docs.google.com/doc..." -O thesis.tex --no-check-certificate
"path/to/pdflatex.exe" -interaction=nonstopmode "thesis.tex"

Those two lines are basically all it takes. When I run build_thesis.bat it downloads the very latest code and compiles it into a pdf.

Notes
  • Problems with the UTF-8 BOM sneaking in to the beginning? Try using lualatex.exe instead which supports UTF-8.
  • If you want to split up into many .tex files you can make many docs and wget all of them.
  • If you use images it's easiest to simply have them in your working folder.
  • Images in the Google Doc will be ignored in the raw text, so you can have your images in the doc and the code for LaTeX to compile them in for a nice preview.
  • A ToC in the doc follow as text in raw format, so it's best not to use it and instead rely on scrolling.
Tell me your results. Happy LaTeXing!

/Johan

Monday, August 8, 2011

SSH med RSA-nycklar


Översikt

För att börja använda RSA-nycklar (istället för)/(tilsammans med) lösenord för SSH behöver du först göra två saker:
  1. Generera ett nyckel-par med PuTTYgen
  2. Klistra in din publika nyckel på servern i $HOME/.ssh/authorized_keys
  3. Lägg in din privata nyckel i PuTTY och spara anslutningen så du slipper göra om det 

Guide för Windows

Se till att du har både PuTTY och PuTTYgenLadda ner här.
Öppna PuTTYgen och generera ett nyckel-par med som är minst 1024 bitar.

Lägg sedan ditt namn i kommentaren, t.ex "rsa-key-John", och (om du vill) ett lösenord som måste skrivas för att nyckeln ska kunna användas.
Varför ha ett lösenord på nyckeln?


Om en hacker får tag på din privata nyckel måste han/hon först brute-force:a RSA-nyckelns lösenord för att kunna logga in på server. Det ger tid för en administratör att upptäcka läckan och blockera den/de läckta nycklarnas tillgång till servern.

Spara både den publika och privata nyckeln på din dator. All säkerhet hänger på att din privata nyckel hålls hemlig. Kopiera den text som är markerad i bildan ovanför. Det är din publika nyckel som du ska lägga in på servern.
Hantering av din privata nyckel.


Några riktlinjer:
  • Lägg inte nyckeln i Dropbox
  • Skicka inte nyckeln i e-post
  • Ge inte din nyckel till någon annan - generera istället en ny
  • Använd ett USB-minne om du behöver flytta en privat nyckel
  • Vill du ha en nyckel hemma också? Generera en ny och maila den publika nyckeln så du kan lägga in den från kontoret

Sista steget: Kopiera din publika nyckel som står i rutan "Public key for pasting into..", och klistra in den i $HOME/.ssh/authorized_keys på en egen rad.

Lägg in din privata nyckel i PuTTY och spara anslutningen i Session så du slipper göra om det.



Det anses vara helt säkert att ha en publik SSH-server som endast accepterar RSA-nyckel-inloggning - förutsatt att du håller dina privata nycklar privata. Därför bör du definitivt stänga av lösenords-inloggning i /etc/ssh/sshd_config så att alla brute-force attacker blir meningslösa.

Sunday, November 7, 2010

My opinion on the future structure of the web

With XHTML Strict we have separated content and design. Using CSS we have been able to remove much redundancy in the code delivered to browsers. We basically define snippets of design and apply them on an arbitrary amount of elements.

I believe the next step is to separate data and structure, by defining snippets of structure and applying to on arbitrary amount of data.

This can be easily achieved now using jQuery Templates, which at the time of writing is in beta.

Example:

Imagine the comment-section on YouTube or this blog. Each comment might have an image, a timestamp, username, thumbs-up/down and some wrapping. All this XHTML code is repeated for every comment.

A short comment

Using a short YouTube comment as an example, the XHTML code consists of 721 bytes of data (which has been generated in a loop on the server side and transmitted to the browser). Removing all XHTML code from the example comment - leaving only the actual information needed - yields 48 bytes of data. In other words: The XHTML code contained 6.657% information, the rest being redundant structure code which is repeated for each comment.

Visual representation:

Code of the short comment:

<li data-author-viewing="False" data-id="5mm3YzP7OX3U298MLeHm0HolNSkstTdEdfbrhWTEAnI" data-score="0" data-author="SuperWizTech" data-pending="0" data-blocked="False" data-flagged="False" data-removed="False" class="comment current"> <div class="metadata"> <div> <a class="author" href="/user/SuperWizTech" title="SuperWizTech">SuperWizTech</a> </div> <div> <span class="time">4 months ago</span> </div> </div> <div class="content"> <div class="comment-text" dir="ltr"> <p>completly awsome? :D</p> </div> <div class="metadata-inline"> <a class="author" href="/user/SuperWizTech">SuperWizTech</a> <span class="time">4 months ago</span> </div> </div> </li>

A long comment

The same comparison on a (relatively) long comment yields an information-to-structure-code-ratio of 453 / 1518 = 29.84%

Visual representation

Code of a long comment

<li data-author-viewing="False" data-id="5mm3YzP7OX2-iXk_QNRtPTEnt5jOC_kWffObtTXjW24" data-score="4" data-author="jpsieben7" data-pending="0" data-blocked="False" data-flagged="False" data-removed="False" class="comment current"> <div class="metadata"> <div> <a class="author" href="/user/jpsieben7" title="jpsieben7">jpsieben7</a> </div> <div> <span class="time">2 years ago</span> <span class="comments-rating-positive">4 <img class="master-sprite comments-rating-thumbs-up" src="http://s.ytimg.com/yt/img/pixel-vfl3z5WfW.gif"></span> </div> </div> <div class="content"> <div class="comment-text" dir="ltr"> <p>what you could do is get a rechargeable battery pack and 2 nxts. then have the base hooked up to the wall so it automatically keeps charging and then have the base use the light sensore and send out a strobe which the bartender finds and? goes and gets. a little bit more complicated but would be a bit more acurate and quick. also make the base have a gravity fed drink loader in it so when 1 is taken another takes its place.</p> </div> <div class="metadata-inline"> <a class="author" href="/user/jpsieben7">jpsieben7</a> <span class="time">2 years ago</span> <span class="comments-rating-positive">4 <img class="master-sprite comments-rating-thumbs-up" src="http://s.ytimg.com/yt/img/pixel-vfl3z5WfW.gif"></span> </div> </div> </li>

Summary

Assuming (educated guesses):

  • Average comment text size: 200 bytes (justified for some Unicode encoding)
  • Additional unique data per comment: 50 bytes (such as: index; special class-names)
  • Data structure overhead per comment: 30 bytes (using JSON)

If we were able to represent the data (username, comment-text and timestamp) with said overhead and have its structure defined only once, the comment section on YouTube would only use less than 25% (280 / 1265 = 22.134% in this example) of it's current bandwidth for the comment section for each [page view]/[comment page-flip].

Simply put, separating structure from data on the web would first of all reduce many chunks of redundant code in repetitive sections (such as in comments- and search-result-sections). Secondly the data would - by design - be set free from their client-specific (in this case client = web-browser) delivery.

One benefit from such a separation would be the possibility of easily creating a completely different interaction to the internet service on a mobile client, without the need of making a new API for this.

Sunday, September 13, 2009

"Spotify Friends" Part 2

Part 2. Controlling Spotify playback by injecting keyboard messages

As we disscussed in Part 1, a Windows application has a WndProc that receives messages such as WM_KEYDOWN from the operating system when the user interacts with the window. Now we want to send those messages ourselves to simulate the user pressing shortcuts (such as [space] for play/pause).

We first prepare by spying on the Spotify window with Spy++ (a tool that comes with Microsoft Visual Studio). Attach to Spotify using Spy++ and filter out just the keyboard messages. Now when you press the shortcut keys in Spotify you will see exactly what messages were sent from the OS. This happens when we press [space]:



To send our own [space] message we do the following:
  • Find the Handle of the Spotify window
  • Send the message using PostMessage( .. ) in user32.dll
Finding the Handle (and getting current song) of the Spotify window in C#

We now search for all processes called Spotify. When we find the right process we can get both the handle we are looking for and the title of the main window - which shows what song (if any) is currently playing. If we cannot find the process then we assume Spotify is not running.

spotifyWindow = 0;

Process[] processes = Process.GetProcessesByName("Spotify");
foreach (Process proc in processes)
{
    if (!proc.MainModule.FileName.EndsWith("spotify.exe")) continue;

    if (proc.MainWindowTitle.Length >= 10)
    {
        NowPlaying = proc.MainWindowTitle.Substring(10);
        SpotifyStatus = AppStatus.Playing;
    }
    else
    {
        NowPlaying = "";
        SpotifyStatus = AppStatus.Paused;
    }

    spotifyWindow = (int)proc.MainWindowHandle;
}

if (spotifyWindow == (int)IntPtr.Zero)
    SpotifyStatus = AppStatus.NotRunning;


Sending messages to the Spotify window

Sending [space] press is just a matter of copycatting the messages we saw in Spy++. But first we need to import the PostMessage( .. ) from user32.dll.
[DllImport("USER32.DLL")]

static extern int PostMessage(

    int hwnd,

    int msg,

    int character,

    uint count);



If you are in a WPF project this is a good time to - like I said at the end of Part 1 - add a reference to System.Windows.Forms and add:

using FormsNS = System.Windows.Forms

If you have a Form application you just use Keys directly. To send [space] we do this:
const int WM_KEYDOWN = 0x100;

const int WM_KEYUP = 0x101;

if (spotifyWindow != 0)

{

     PostMessage(spotifyWindow, WM_KEYDOWN, (int)FormsNS.Keys.Space, 1);

     Thread.Sleep(100);

     PostMessage(spotifyWindow, WM_KEYUP, (int)FormsNS.Keys.Space, 1);

}

This time we get away with putting any integer as the last argument in PostMessage, but we will have to be more precise later on when we are sending the more complex shortcuts (like Ctrl+Right).

Sending Ctrl + Key to Spotify

If we look in Spy++ it seems like we can get away with simulating both the [ctrl] and the [right] key at the same time, but that will not work. My theory is that when Spotify gets the [right] key it will look at the keyboard to see if [ctrl] is pressed (because this is easier to do). Our messages so far will not fool Spotify if it looks at the state of the keyboard - so we now need to choose one out of two solutions:
  1. Force ourselves to use [ctrl] + [any other keys] in our global hot-key combo (Spotify will see that [ctrl] is pressed when we send [right] for example)

    or
  2. Use other key combos, and fake that [ctrl] is pressed
If you are satisfied with choice 1 you just need to send the [right] (etc.) keys correctly and you are done. For that reason we will first look at how we send those keys, then look at how we can spoof the [ctrl] key.


Sending [up], [down], [left] or [right]

This time we need to get the last argument in PostMessage correct. We press [ctrl] + [right] in Spotify while spying with Spy++. Then we right-click and look at the Properties on the WM_KEYDOWN message that has VK_RIGHT in it (should be the second one if you cleared the log). In Properties we first see wParam which is the code for the key - (int)FormNS.Keys.Right in this example. We also see the lParam which we need to send as the last argument in PostMessage.

But I have already done that work for you. Here you go:
PostMessage(spotifyWindow, WM_KEYDOWN, (int)FormsNS.Keys.Right, 0x014D0001);
PostMessage(spotifyWindow, WM_KEYUP, (int)FormsNS.Keys.Right, 0xC14D0001);

PostMessage(spotifyWindow, WM_KEYDOWN, (int)FormsNS.Keys.Left, 0x014B0001);
PostMessage(spotifyWindow, WM_KEYUP, (int)FormsNS.Keys.Left, 0xC14B0001);

PostMessage(spotifyWindow, WM_KEYDOWN, (int)FormsNS.Keys.Up, 0x01480001);
PostMessage(spotifyWindow, WM_KEYUP, (int)FormsNS.Keys.Up, 0xC1480001);

PostMessage(spotifyWindow, WM_KEYDOWN, (int)FormsNS.Keys.Down, 0x01500001);
PostMessage(spotifyWindow, WM_KEYUP, (int)FormsNS.Keys.Down, 0xC1500001);



NOTE: Put in a Thread.Sleep(100) between each key you send. I guess the application will not react if the keys are pressed for an extremely short timespan.
If you choose to have [ctrl] in your global hot-keys you now know all you need to know to control Spotify's playback, volume and get the currently playing song.


Faking the [ctrl] key being pressed

To set the [ctrl] key as down or up we need to import the following method in user32.dll:
[DllImport("user32.dll", EntryPoint = "keybd_event", CharSet = CharSet.Auto, ExactSpelling = true)]

static extern void keybd_event(byte vk, byte scan, int flags, int extrainfo);

To press down we pull this one-liner:
keybd_event((byte)FormsNS.Keys.ControlKey, 0x1D, 1, 0);


And to release [ctrl]:
keybd_event((byte)FormsNS.Keys.ControlKey, 0x1D, 1 | 2, 0);

Using these you can now have global hot-keys such as [alt] + [up] (volume up) and [alt] + [right] (next song). When Spotify checks to see the status of the [ctrl] key it will see it as being pressed.

Summary

Looking back we can sum up what we need to do to send key-presses to another window:
  • Get the Handle of the window. The Handle is used by the Windows GUI manager to refer to uniquely identify windows (and other controls).
  • Send the correct messages via PostMessage. Note that the last argument might be important. In that case we can easily get it with Spy++.
  • If we do not want to hold down [ctrl], but we want the application to think [ctrl] is being held down, then we probably need to use keyd_event( .. ) to set [ctrl]

Saturday, September 12, 2009

C# / WPF project: "Spotify Friends" Part 1

Let's kick it off with a mixture of C# with WPF and (kind of) C++.
I'll start by saying; this project is in no way associated with or endorsed by the amazingly great music service Spotify. This project has no intent of misusing the Spotify trademark or in any other way step on anyones toes. "Spotify Friends" is just an internal code-name.

Overview


"Spotify Friends" is a [client(s) --> server] project that lets users see what their friends are listening to at the moment. The client also adds the functionallity of controlling the playback with global hot-keys; changing the song or volume while playing games or surfing - without opening the Spotify window.


Part 1. Global keyboard hooking in C# Forms and WPF


How it's done in C++


If we were to make a Windows GUI application in C++, we would have a method in our GUI-class called WndProc which would receive messages from the operating system, such as WM_KEYDOWN or WM_LBUTTONDOWN (left button down). The WM_KEYDOWN message will only come to us when the user pushes a button and our window is the currently active one, so we can't rely on that one for our global hot-keying. Luckily the people a Microsoft created just what we need; a method in user32.dll called RegisterHotKey( .. ) that allows us to ask the OS:  "Can you please notify me any time the user presses Alt + Z? :)".  If no other application has been granted that combination, the OS will gladly say yes. So from now on when the user presses Alt + Z, our WndProc will receive a message called WM_HOTKEY and parameters explaining which key combination was pressed.


Registering for hot-keys in C#


We need to run the RegisterHotKey method in user32.dll. To get access to this method we import it like this:







// Register a hot key with Windows
[DllImport("user32.dll")]
private static extern bool RegisterHotKey(IntPtr windowHandle, int id, uint modifiers, uint virtualKey);

// Unregister a certain hot key with Windows
[DllImport("user32.dll")]
private static extern bool UnregisterHotKey(IntPtr windowHandle, int id);




To register a hot-key we must have a handle to our window.

Getting the Handle to our window in a Windows.Form application:

IntPtr handle = this.Handle;





Getting the Handle to our window in a WPF application:

IntPtr handle = new WindowInteropHelper(this).Handle;

Now when we have the handle we can register a hot-key when our window has loaded. The id we use when registering is used later for unregistering. Here is a simple example of registering:

// Register a hot key


if (!RegisterHotKey(handle, 1, (uint) (ModifierKeys.Alt | ModifierKeys.Control), (uint) 'P'))
throw new InvalidOperationException("Could not register the hot key");


Listening to messages in a Windows.Form application


In C# we can create a Windows.Forms application. This is the old school way of creating a GUI application, which was the only way back in the days of .NET 2.x. In a Form class we can override the WndProc and get direct access to the messages that the OS is sending to our window. We can now listen to the WM_HOTKEY message and react accordingly.


Alright, how is this done in WPF?


WPF is very different from Windows.Form and doesn't have a WndProc method. But we will make our own. Let's call it WndProc.



const int WM_HOTKEY = 0x0312;






public IntPtr WndProc(IntPtr hwnd, int msg, IntPtr wParam, IntPtr lParam, ref bool handled)
{
    // check if we got a hot key pressed.
    if (msg == WM_HOTKEY)
    {
        // get the keys.
        uint key = (uint)(((int)lParam >> 16) & 0xFFFF);
        ModifierKeys modifier = (ModifierKeys)((int)lParam & 0xFFFF);

        // invoke the event to notify the parent.
        KeyPressed(modifier, key);
    }

    return IntPtr.Zero;
}



Now we just ask an underlying mechanism of WPF to kindly send us all the messages. We can preferably do this when the window has loaded.

HwndSource src = HwndSource.FromHwnd(new WindowInteropHelper(this).Handle); 



src.AddHook(new HwndSourceHook(WndProc));












Summary of global hot-key listening


We have now done the following:
  • Imported the RegisterHotKey( .. ) method from user32.dll
  • Used it to register a hot-key combo i.e Ctrl+Shift+P
  • Getting our code to parse messages in WndProc
    • In Windows.Form: By overriding WndProc( .. )
    • In WPF: By creating WndProc( .. ) and adding a hook
These are the main steps, and our application should now be reacting to the global hot-keys that we want.

Sources and tips

Thanks to Christian Liensberger for his article on hooks in C#.
Thanks to softwerx for his response showing how to use WndProc in WPF.
The Keys enum in Windows.Forms can be used when registering and parsing keys. In a WPF project one can add a reference to System.Windows.Forms and then add:

using FormsNS = System.Windows.Forms;




This will avoid conflicts with MessageBox for example in WPF vs. Forms. We can now easily register and keys like this:

// Register a hot key



if (!RegisterHotKey(handle, 1, (uint)(ModifierKeys.Alt | ModifierKeys.Control), (uint)FormsNS.Keys.Down))
throw new InvalidOperationException("Could not register the hot key");

This way we don't need to define "virtual key-codes", since they correspond to the uint value of keys in Keys.