Using Multiple Network Interfaces simultaneously under Linux (Virtual/Real)

Scenario

My system is connected to a college hostel LAN. Internet is provided through the Cyberoam system. Thus, there is a cap on the maximum transfer speed for internet. I had an idea since months to exploit virtual interfaces to get more out of it. For eg, parallel downloads all at the max transfer speed and browsing simultaneously without being affected by those downloads. I didn’t find any favourable ways to achieve this until last couple of days when I came across various pieces which I could put together to get it done.

The whole idea is to create a virtual ethernet interface (which is connected to the LAN) using ip aliasing and route selected traffic through that interface in a simple way. We will be using a proxy server to bind connections to that ip alias as it is much more simpler to configure any application to use a particular proxy server than to set their outgoing interface.

Although I will considering only virtual interfaces in the tutorial, the same technique can be applied to actual multiple interfaces as well. Also, note that there are tons of ways one might want to configure the behaviour. This is another reason why I will be using a proxy server.

Setting up the virtual interface

The network I am connected to is 192.168.124.0/255.255.252.0. Thus accordingly, create a virtual interface with ip 192.168.124.122

$ sudo ifconfig eth0:1 192.168.124.122 netmask 255.255.252.0 broadcast 192.168.127.255

eth0:1 is the new virtual interface. It can be anything like eth0:3 or eth0:25. Also you can make a lot of them.

Testing it out

Just make a curl request to a destination using --interface

$ curl --interface "192.168.124.122" 192.168.125.63:8000

192.168.125.63 is my eth0 address and I am just making a curl request to my http server to check the source ip is indeed the aliased one. You can make request to anything reachable to the network.

I do all my downloading through scripts which use curl or wget, so I can theoretically modify them to use --interface, but it is too much work and I need most applications to be able to work this way. Thus, enter proxy.

Setting up the proxy server

We will be using squid to create our proxy server mainly because it supports tcp_outgoing_interface directive which lets us select outgoing interface using squid acl rules. That way, it will be easier when number of virtual interfaces we wish to use increases.

Install latest squid from your distribution repository. Edit /etc//etc/squid/squid.conf

http_port 3128
http_access allow localhost
dns_v4_first on

tcp_outgoing_address 192.168.124.122

For the sake of simplicity, I have kept the configuration minimal. The proxy runs at port 3128, and access is allowed to localhost. There is a lot that can be done with squid, so I will highly recommend going through the squid documentation and tutorials around the internet.

dns_v4_first on is used to make sure squid uses ipv4 first when contacting servers and peers. Befire this, I spend couple days wondering why I couldn’t make HTTPS connections. After frustrating troubleshooting with logs, wireshark, curl, and whatnot, this directive solved my problem. Most systems will be fine without this directive, so it can be removed.

tcp_outgoing_address 192.168.124.122 binds all outgoing connections to the aliased ip, which was our goal to begin with. This simply applies this to all the outgoing connections through the server. You may, use it to do more powerful filtering. Go though the tcp_outgoing_address documentation.

Start the squid daemon with (depending on your distribution)

$ sudo systemctl start squid

Using the proxy

All the work is done. Just use the http proxy localhost:3128 for your application. Most applications respect the environment variables for proxy. So if you want a terminal session to launch programs which connect to the network using the second ip, do the following

export http_proxy=http://localhost:3128/
export https_proxy=$http_proxy
export ftp_proxy=$http_proxy
export rsync_proxy=$http_proxy
export no_proxy="localhost,127.0.0.1"

For browsing you may set the proxy for firefox directly through the settings. For chrome/chromium, use chromium --proxy-server="http://localhost:3128"

Extending to more than two interfaces

You just need to use tcp_outgoing_address according to some parameter like port or ip. You will need to make a different port for each extra interface and write a tcp_outgoing_address for each port.

Feedback, Problems, or any Alternative or better way? Throw them up on the comments.

IPod Shuffle 4g under Linux

Couple of months ago, I got an Apple Ipod Shuffle 4g. Despite the presence of IPod syncing apps under Linux, my experience was annoying. Shuffle 4g changed compared to its predecessors regarding the database format and new features, so a lot of plugins and apps don’t work as expected. In this post, I will write about different ways to tackle this annoyance.

Basic Things to Know

Bah! why can’t Apple make straightforward stuffs! Anyway, here are things you should know in a nutshell. IPod’s internal software works on a defined structure in the mass storage of the IPod. The process of creating this structure may be referred to as initializing your device. For adding any content to your device, a simple copy is not enough, changes need to be made in the database (The iTunesSD file) as well.

If you want to know more about the database format, I highly encourage you to read it at http://shuffle3db.wikispaces.com/iTunesSD3gen

Some other ways to use Shuffle 4g under Linux (which I didn’t prefer)

  • gtkPod: The most popular iPod syncing app under Linux. Seemed too complex initially. Random crashes made it more annoying. Although it works, lacks the feature for voiceover.
  • Floola: Didn’t work at all with my IPod + Crashes!
  • Rhythmbox with IPod Plugin: Rhythmbox wasn’t able to initialize my IPod and I had to do it using gnupod. Other than this, it works fine. It took me a bit of initial effort to get my IPod recognized by Rhythmbox. I had somehow screwed up the partition structure of my IPod. Anyway, if you face the same problem, delete the whole IPod partition and repair it with iTunes. The plugin however lacks the voiceover feature.

My preferred way, using shuffle-db-ng

I stumbled across this modest looking project at https://code.google.com/p/shuffle-db-ng/ and took a liking to it immediately. It is a command line tool written in python and is simple and straightforward. And yes, it supports voiceover too!

  • Put your music files in the IPod
  • Run the Script, Done!

I studied the whole script after reading about the database format and learned few cool things. I wonder why the voiceover support is still absent from other popular IPod apps.

As a prerequisite, you need the following

  • python2.7
  • mutagen python library (Required for accessing audio tags)
  • pico2wav for voiceover audio (can be found in svox pico package. For Arch Linux users)

You may use the original shuffle-db-ng, or use my modified version which has following additions

  • Automatic initialization of the directory tree
  • Option to disable voiceover to save space
  • Renames file which crash the program raising Unicode Error (using the –rename-unicode flag)
  • A usage message on flags ‘-h’, ‘–help’

I will use here my version (obviously ;))

~ $ https://github.com/nims11/IPod-Shuffle-4g.git
~ $ cd IPod-Shuffle-4g/
~/IPod-Shuffle-4g $ python shuffle.py -h
usage: shuffle.py [-h] [--disable-voiceover] [--rename-unicode] path

positional arguments:
path

optional arguments:
-h, --help           show this help message and exit
--disable-voiceover  Disable Voiceover Feature
--rename-unicode     Rename Files Causing Unicode Errors, will do minimal
required renaming

# Lets copy few music files to my ipod mounted at /path/to/ipod
~/IPod-Shuffle-4g $ cp ~/Music/*.mp3 /path/to/ipod
~/IPod-Shuffle-4g $ python shuffle.py /path/to/ipod
[*] Adding track /path/to/ipod/01 - Bitter Sweet Symphony.mp3
[*] Adding track /path/to/ipod/01 Ace Of Spades.mp3
[*] Adding track /path/to/ipod/02 - Black Sabbath - Iron Man.mp3
[*] Adding track /path/to/ipod/11 - Angel of Death.mp3
[*] Adding track /path/to/ipod/Center Of The Universe.mp3
[*] Adding track /path/to/ipod/Vodka.mp3
[+] Adding master playlist

# Unmount and Unplug your IPod and test it out!

But with the limited capacity of this little device, adding music this way will be tiring. Thanks to the python script, you can treat your IPod as a normal mass storage MP3 device, sync into it using rhythmbox or any other player, and run the shuffle.py to build the database.

To recognize IPod as an media device, add a file named .is_audio_player to the root of your IPod with the following content

name="VV's IPOD"
audio_folders=Music/

Open Rhythmbox or any other player, disable the IPod plugin and make sure the MTP plugin is enabled. Search for new devices and your IPod storage will show up. Add whatever you want to and before ejecting your IPod, run python shuffle.py /path/to/ipod as mentioned in the example.

shuffle.py also supports playlists (m3u and pls), but they are a bit of a hassle. The files pointed by the playlist should be inside the ipod storage. If this is true, shuffle.py will automatically scan the playlist file and build the database. Rhythmbox saves the day again! Open the MTP device in Rhythmbox. Select the playlists to sync through Your Device -> properties -> sync, select your playlists, then press the sync button. The .pls files will be created on your IPod and next time you run shuffle.py, Viola! you have your playlists on your IPod!

a

NOTE: In Rhythmbox, the playlist syncing can sometimes produce incorrect .pls files. I observed that this happens when there is some difference in the ID3 tags like artist and album artist in which case the path specified in the .pls file by the MTP plugin sometimes goes wrong, causing the shuffle.py to crash! In such cases, close Rhythmbox, delete the .pls files (not audio files) created on your IPod, start Rhythmbox, sync your device again, the hopefully the new playlists should be right.

If some other player works fine or better, please do comment :)

Further Development

I wish to do the following in my free time

  • Inbuilt support for adding playlists from hard drives.
  • A last.fm scrobbler using the info here
  • A Qt frontend (maybe)

Any suggestions, doubts, feedback on this post? Please do comment!

Handling constantly Interrupting Downloads using curl

My internet connection has an annoying problem of interrupting downloads too frequently. That way, downloading even resumable files can be tedious. Using the following one liner with curl, Life becomes much easier


until curl -C - -o "<output_file>" "<url>"; do :; done

Until makes sure the loop continues till curl returns a success status. A colon(:) is  a no operation command. This one liner can obviously be modified to suit your needs.

Moving to Nightingale

I had been using Rhythmbox since a long time. It was perfect for me, with clean interface, synced well with my Phone, had Last.fm support and lyrics support. However, encountered Nightingale a day ago and completely fell in love with it to extent I am switching permanently to it.

Nightingale

Nightingale forked off from Songbird, when it discontinued its support for Linux. My first reaction playing with Nightingale was, “Firefox modded into a music player?”. Nightingale’s engine is based on Mozilla’s XULRunner and Firefox users will feel home.

UI: The first thing impressed me about Nightingale was its beautiful and sleek UI. Most music players with sleek and attractive UI usually have a trade off with functionality. And most players with too much functionality usually end up being ugly. Nightingale puts up lots of things in front of you while maintaining its beauty and simplicity.

UI

Plugins: All plugins I used in Rhythmbox have a equally good, if not a better counter part in Nightingale. Unlike most players, installing add-ons is as simple (same) as in Firefox. Existing extensions will most probably have you covered. You have a Last.fm scrobbler, mashTape, SHOUTcast, MLyrics, Guitar Tab etc. Add-on architecture is solid  and as Nightingale becomes popular, more 3rd party plugins will pour in.

Installing Addons

Installing Addons

SHOUTcasy

SHOUTcast

Library: It is very similar to Rhythmbox, and it is a good thing!

lib

Extras: A builtin web browser, support for Video playback, sync, skins (Feathers).

What Sucks?

Bugs: Although it seems perfect for daily use, random crashes are not rare. I felt the git version was a lot more bug free, but switched back to release version as Last.fm didn’t seem to work.

System Resources: Stats show a little more system load than Rhythmbox in my case. However, the difference is not really big. Also, I don’t get heavy disk usage with Nightingale which I used to face in Rhythmbox.

Media Keys don’t work out of the box: Some obvious workarounds are required to make it run, but an out of box support would be great.

Importing your Rhythmbox playlist

Unfortunately, there isn’t a point and click way. I did it the following way

  1. Open ~/.local/share/rhythmbox/playlists.xml
  2. Copy the contents inside your desired <playlist> tag to a new file, say xyz.m3u
  3. Remove all <location> and </location>. Under vim, this is as simple as :%s/<location>//g and :%s/<\/location>//g
  4. Import the m3u file into Nightingale.

Running Dota (1) on Linux with full customkeys

I have been running DoTA conveniently under Linux using Wine, since more than an year when I actually started playing DoTA. Everything works perfect without much effort. In this post, I explain the basic steps to get full DoTA Experience.

Configuring Wine

Very little configuration required. Might run straightaway. But there might be problems which will prevent you from switching between windows, minimizing, lose window focus, taskbar showing on top, etc. The best way to solve these is to configure wine to emulate a virtual desktop, and launch Frozen Thrones in a separate workspace. That way, you can switch between workspace conveniently using keyboard shortcuts (It is more convenient than minimizing under Windows). Execute winecfg and select Emulate Virtual Desktop under the Graphics tab. Make sure to fill up your native resolution there (or the resolution at which you will run the game). Run the game through


nimesh Warcraft III  1.26 $  wine Frozen\ Throne.exe

or

nimesh Warcraft III  1.26 $  wine Frozen\ Throne.exe -opengl  # If the above doesn't work for you well.

One problem that I regularly face while switching windows is that the Alt key becomes sticky, causing signalling in the map whenever I click somewhere. Keep Calm, and press Alt once again and release. This will solve it.

Customkeys for Skills

Unfortunately, Warkeys and others of its kind doesn’t work with wine. But there is an alternative for Customizing keys for skills. Head towards drjones.dk/customkeys/generator.php, generate your CustomKeys.txt and paste it under Warcraft III folder (along with Frozen Throne.exe) as CustomKeys,txt. The item section in the website only works with the CustomKey client, which doesn’t work with wine.

Customkeys for Items

Time for a confession. Why would I write a guide on running DoTA now after more than a year I started playing it? Because recently I wrote a tool for mapping item keys for DoTA which runs under Linux. Clicking items with mouse is a pain, so I came up with this nifty tool using xmodmap and xbindkeys.


nimesh ~ $  git clone https://github.com/nims11/KeyMapper.git
Cloning into 'KeyMapper'...
remote: Counting objects: 14, done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 14 (delta 2), reused 14 (delta 2)
Unpacking objects: 100% (14/14), done.
nimesh ~ $  cd KeyMapper/
nimesh ~/KeyMapper $  chmod +x *
nimesh ~/KeyMapper $  ./KeyMapper  # Start keymapper, press anywhere F7 Toggle Custom Key Mappings.

Read up the README and keys.conf file. Its really simple, and got you covered.

Enjoy!
dota

Using Hostapd with dnsmasq to create Virtual Wifi Access Point in Linux

NOTE: This is an alternative branch off from my previous Hostapd guide, which I really recommend going through before this.

In my previous hostapd guide, I used dhcpd to assign IP addresses to the clients connecting to the access point. While this works fine for most scenarios, it is an overkill to use dhcpd for such situations where normally the number of clients is 2-3, or around 20 at max. For such cases, dnsmasq is a better option.

Installing

Install dnsmasq from somewhere

# Arch Linux
sudo pacman -S dnsmasq
# Ubuntu
sudo apt-get install dnsmasq

Configuring dnsmasq

The main reason I am recommending dnsmasq over dhcpd is the ease in configuring it. Less hassle in configuration means less problems and better troubleshooting. Most of the problems users faced in my previous guide was dhcpd related.

The default /etc/dnsmasq.conf explains all its configuration options pretty well, so I will jump straight to what your /etc/dnsmasq.conf should look like.

Just append the following to the /etc/dnsmasq.conf

# disables dnsmasq reading any other files like /etc/resolv.conf for nameservers
no-resolv
# Interface to bind to
interface=wlan0
# Specify starting_range,end_range,lease_time
dhcp-range=10.0.0.3,10.0.0.20,12h
# dns addresses to send to the clients
server=8.8.8.8
server=8.8.4.4

Simple, isn’t it?

Final Steps

The final steps involves enabling NAT to share internet in one network interface  with the clients connected through hostapd.
I have included all the steps to configure wlan interface, enable NAT, start dnsmasq and hostapd in the BASH script below
Let the name of this file be initSoftAP
Copy the content below to the file initSoftAP  (Perform changes if required)

#!/bin/bash
#Initial wifi interface configuration
ifconfig $1 up 10.0.0.1 netmask 255.255.255.0
sleep 2

###########Start dnsmasq, modify if required##########
if [ -z "$(ps -e | grep dnsmasq)" ]
then
 dnsmasq
fi
###########

#Enable NAT
iptables --flush
iptables --table nat --flush
iptables --delete-chain
iptables --table nat --delete-chain
iptables --table nat --append POSTROUTING --out-interface $2 -j MASQUERADE
iptables --append FORWARD --in-interface $1 -j ACCEPT

#Thanks to lorenzo
#Uncomment the line below if facing problems while sharing PPPoE, see lorenzo's comment for more details
#iptables -I FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

sysctl -w net.ipv4.ip_forward=1

#start hostapd
hostapd /etc/hostapd/hostapd.conf 1> /dev/null
killall dnsmasq

It might be more convenient to use hostapd -B /etc/hostapd/hostapd.conf which runs hostapd in background, but take care of the ‘killall dnsmasq’ if you choose this option. (Thanks to Enda for pointing out)

Make this file executable, and run it. The syntax for executing it is

./initSoftAP wifi_card_interface interface_with_internet

chmod +x initSoftAP
./initSoftAP wlan0 eth0  # And there you go

The “wifi_card_interface” will be wlan0 most of the cases. For “interface_with_internet“, since I want to share internet from my ethernet network interface, I used eth0. If I ever want to share internet from my 3g modem, I use ppp0. (These values need not be same for everyone)
You may see available network interfaces by

ifconfig -a

That’s all folks!
Problems, Errors, Feedback or any alternatives? Feel free to reply.

Downgrading Packages in Arch Linux: The Worst Case Scenario

Nonsense Part (You may Skip it)

Downgrading packages is a thing i don’t often encounter while using Arch. If something breaks, i wait for it to get fixed and the update again. But this time, the case was different. Late night, after a kernel (or libnl, i don’t remember) upgrade, my hostapd stopped working, complaining about shared libraries. Shockingly, i discovered that my hostapd was still at 0.7 while the latest stable version was at 2.0!(Reason provided at end) So i did an pacman -Sc to remove older hostapd versions from cache, which didn’t change anything, then made some correct changes and upgraded hostapd to 2.0. New problems in hostapd popped up, about nl80211 being unable to set my wlan to master mode. Now ahead of me was option of spending the rest of night desperately debugging the problem or study for the exam coming morning. Hostapd not working is a more serious issue for me.
Continue reading