Nginx and git via HTTP

Nginx is a high-performance HTTP server and proxy that has been gaining a lot of traction recently. It’s easy to set up, can handle a large number of clients with minimal resources, modular and supports SSL/TLS. Dynamic web content can be served via FastCGI to Nginx.

Git uses WebDAV for pushing via HTTP(S). Even though Nginx handles WebDAV requests, not all DAV commands are implemented in its DAV module. Locking and PROPFIND are missing in particular, and since these commands are used by Git when pushing via HTTP, pushing to a plain Nginx server is not possible at the moment.

One solution is to run Apache on a different port and proxy Git requests to it. However, Apache is such a resource hog that if a lightweight solution is desired it’s out of question. Here comes PyWebDAV, a standalone WebDAV server written in Python, into the picture.

Continue reading

Python-iptables

Python-iptables is a Python project that provides bindings to the iptables C libraries in Linux. Interoperability with iptables is achieved using the iptables C libraries (libiptc, libxtables, and iptables extensions), not calling the iptables executable and parsing its output as most other iptables wrapper libraries do; this makes python-iptables faster and not prone to parsing errors, at the same time leveraging all available iptables match and target extensions without further work. If you need further information on how iptables and netfilter work, this is a great summary.

Continue reading

New Android Market countries

On 11th May 2011 Google announced the availability of paid apps in 99 new countries.

Continue reading

Android emoticons on Ubuntu with Empathy

The default Ubuntu IM client has now been Empathy for a while. Though it supports themes, replacing the stock emoticons (which come from Gnome), unlike in Pidgin, is not a straightforward process. Which is kind of embarrassing, since the default smileys in Gnome are far from being pretty and eye-candy.

Fortunately, you can override the system emoticons for your user: just create the proper directory structure under ~/.local, and applications will use the emoticons found under ~/.local/share/icons/gnome/16×16/emotes, where 16×16 is the resolution they need.

To make it easy for Ubuntu users, we just packed the Android emoticons into a ready to use package. Once you downloaded it, untar the package into your home folder:

$ cd ~
$ tar xvzf ~/Downloads/android-emoticons.tar.gz

It will create all the necessary folders under .local, and place the smileys in different resolutions there.

Implementing SMTP or IMAP XOAUTH authentication in Java

XOAUTH is a SASL authentication mechanism that can be used with the IMAP AUTHENTICATE and SMTP AUTH commands. It is based on OAuth, and uses the same OAuth parameters to authenticate against an SMTP or IMAP server. It is backed by Google, and you can use XOAUTH with Gmail SMTP or IMAP to authenticate without an username and its corresponding password.

You need to go through the usual “OAuth dance” to acquire an access token and its corresponding secret. Once you have that, you can generate the OAuth signature the usual OAuth way, with the following parameters:

  1. The HTTP method is GET,
  2. The URL has the following form: https://mail.google.com/mail/b/[user email address]/[protocol]/, where protocol is either “smtp” or “imap” (without quotes) and
  3. All the usual OAuth parameters except “oauth_signature”.

The actual XOAUTH string will be constructed from the HTTP method, the URL as defined above and the OAuth parameters, the three parts concatenated with space characters, and then base64-encoded. You can find examples for it at Google’s reference page.

We can build the XOAUTH string in a straightforward way with signpost and our Android OAuth helper class (if you’d like to use it in plain Java, just remove the Log lines). Since many people have asked for the source code of the class, here it goes the whole stuff as used in SMSForwarder.


package com.nilvec.oauth;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import java.util.SortedSet;
import java.util.Map.Entry;

import org.apache.commons.codec.binary.Base64;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;

import android.util.Log;

import oauth.signpost.OAuth;
import oauth.signpost.OAuthConsumer;
import oauth.signpost.OAuthProvider;
import oauth.signpost.commonshttp.CommonsHttpOAuthConsumer;
import oauth.signpost.commonshttp.CommonsHttpOAuthProvider;
import oauth.signpost.commonshttp.HttpRequestAdapter;
import oauth.signpost.exception.OAuthCommunicationException;
import oauth.signpost.exception.OAuthExpectationFailedException;
import oauth.signpost.exception.OAuthMessageSignerException;
import oauth.signpost.exception.OAuthNotAuthorizedException;
import oauth.signpost.http.HttpParameters;
import oauth.signpost.signature.HmacSha1MessageSigner;
import oauth.signpost.signature.OAuthMessageSigner;

public class OAuthHelper {

    private static final String TAG = "OAuthHelper";

    private OAuthConsumer mConsumer;
    private OAuthProvider mProvider;

    private String mCallbackUrl;

    public OAuthHelper(String consumerKey, String consumerSecret,
            String scope, String callbackUrl, String appname)
    throws UnsupportedEncodingException {
        String reqUrl;
        if (appname == null)
            reqUrl = OAuth.addQueryParameters(
                    "https://www.google.com/accounts/OAuthGetRequestToken",
                    "scope", scope);
        else
            reqUrl = OAuth.addQueryParameters(
                    "https://www.google.com/accounts/OAuthGetRequestToken",
                    "scope", scope, "xoauth_displayname", appname);

        mConsumer = new CommonsHttpOAuthConsumer(consumerKey, consumerSecret);

        mProvider = new CommonsHttpOAuthProvider(reqUrl,
                "https://www.google.com/accounts/OAuthGetAccessToken",
                "https://www.google.com/accounts/OAuthAuthorizeToken?hd=default");
        mProvider.setOAuth10a(true);

        mCallbackUrl = (callbackUrl == null ? OAuth.OUT_OF_BAND : callbackUrl);
    }

    public String getRequestToken()
    throws OAuthMessageSignerException, OAuthNotAuthorizedException,
    OAuthExpectationFailedException, OAuthCommunicationException {
        String authUrl =
                mProvider.retrieveRequestToken(mConsumer, mCallbackUrl);
        return authUrl;
    }

    public String[] getAccessToken(String verifier)
    throws OAuthMessageSignerException, OAuthNotAuthorizedException,
    OAuthExpectationFailedException, OAuthCommunicationException {
        mProvider.retrieveAccessToken(mConsumer, verifier);
        return new String[] {
                mConsumer.getToken(), mConsumer.getTokenSecret()
        };
    }

    public String[] getToken() {
        return new String[] {
                mConsumer.getToken(), mConsumer.getTokenSecret()
        };
    }

    public void setToken(String token, String secret) {
        mConsumer.setTokenWithSecret(token, secret);
    }

    public String getUrlContent(String url)
    throws OAuthMessageSignerException, OAuthExpectationFailedException,
    OAuthCommunicationException, IOException {
        HttpGet request = new HttpGet(url);

        // sign the request
        mConsumer.sign(request);

        // send the request
        HttpClient httpClient = new DefaultHttpClient();
        HttpResponse response = httpClient.execute(request);

        // get content
        BufferedReader in = new BufferedReader(
                new InputStreamReader(response.getEntity().getContent()));
        StringBuffer sb = new StringBuffer("");
        String line = "";
        String NL = System.getProperty("line.separator");
        while ((line = in.readLine()) != null)
            sb.append(line + NL);
        in.close();

        return sb.toString();
    }

    public String buildXOAuth(String email) {
        String url =
            String.format("https://mail.google.com/mail/b/%s/smtp/", email);
        HttpRequestAdapter request = new HttpRequestAdapter(new HttpGet(url));

        // Sign the request, the consumer will add any missing parameters
        try {
            mConsumer.sign(request);
        } catch (OAuthMessageSignerException e) {
            Log.e(TAG, "failed to sign xoauth http request " + e);
            return null;
        } catch (OAuthExpectationFailedException e) {
            Log.e(TAG, "failed to sign xoauth http request " + e);
            return null;
        } catch (OAuthCommunicationException e) {
            Log.e(TAG, "failed to sign xoauth http request " + e);
            return null;
        }
        HttpParameters params = mConsumer.getRequestParameters();

        // Since signpost doesn't put the signature into params,
        // we've got to create it again.
        OAuthMessageSigner signer = new HmacSha1MessageSigner();
        signer.setConsumerSecret(mConsumer.getConsumerSecret());
        signer.setTokenSecret(mConsumer.getTokenSecret());
        String signature;
        try {
            signature = signer.sign(request, params);
        } catch (OAuthMessageSignerException e) {
            Log.e(TAG, "invalid oauth request or parameters " + e);
            return null;
        }
        params.put(OAuth.OAUTH_SIGNATURE, OAuth.percentEncode(signature));

        StringBuilder sb = new StringBuilder();
        sb.append("GET ");
        sb.append(url);
        sb.append(" ");
        int i = 0;
        for (Entry<String, SortedSet<String>> entry : params.entrySet()) {
            String key = entry.getKey();
            String value = entry.getValue().first();

            int size = entry.getValue().size();
            if (size != 1)
                Log.d(TAG, "warning: " + key + " has " + size + " values");

            if (i++ != 0)
                sb.append(",");
            sb.append(key);
            sb.append("=\"");
            sb.append(value);
            sb.append("\"");
        }
        Log.d(TAG, "xoauth encoding " + sb);

        Base64 base64 = new Base64();
        try {
            byte[] buf = base64.encode(sb.toString().getBytes("utf-8"));
            return new String(buf, "utf-8");
        } catch (UnsupportedEncodingException e) {
            Log.e(TAG, "invalid string " + sb);
        }

        return null;
    }

}

Don’t forget to provide feedback if you find this class or article useful. Happy hacking!