Git Clean

The majority of my programming projects are tracked using git. It’s decentralized, stable, compact and easy to configure, so it’s an obvious choice as a developer.

It allows me to split off my work into separate, tracked branches so I can work on multiple features or changes at the same time without risking the health of the codebase.

Once I’ve finished a feature in a branch, I can merge it into master and can happily use the new code. But the branch I worked in is still there. After several dozen mini “projects” like this, my git branch list starts getting very long.

Luckily, I found this short command line script to clean up branches that have been merged into master!

git branch --merged master | grep -v "\* master" | xargs -n 1 git branch -d  

This lists all branches that have been merged into master (including master), filters out the master branch, and then runs git branch -d for each branch name found this way.

But wait! If you work with a remote git repository, this still leaves all of those branches out there! Never fear, there’s a (pretty complicated) way to clean those up, too!

git branch -r --merged  master | grep origin | grep -v '>' | grep -v master | xargs -L1 | cut -d"/" -f2- | xargs git push origin --delete  

This lists all remote branches merged to master, finds all of the branches from origin (you can replace origin with your remote destination of choice), filters out branches you don’t want to delete, and then runs git push origin --delete for each branch found this way.

Tada! Sparkling clean git branches locally and remote.

Advertisements
Git Clean

Ubuntu 15.10 Quad Monitor xorg.conf configuration

At work I was tasked with setting up a NOC-esque monitoring wall, using an old desktop, two video cards and 4 monitors. I wiped the box and installed Ubuntu 15.10 on it, and immediately jumped into a world of hurt. First, here’s the final result.

$ cat /etc/X11/xorg.conf
# nvidia-settings: X configuration file generated by nvidia-settings
# nvidia-settings:  version 352.21  (buildd@lgw01-37)  Thu Jul 23 11:50:49 UTC 2015

# nvidia-xconfig: X configuration file generated by nvidia-xconfig
# nvidia-xconfig:  version 340.96  (buildmeister@swio-display-x86-rhel47-05)  Sun Nov  8 22:50:27 PST 2015

Section "ServerLayout"
    Identifier     "Layout0"
    Screen      0  "Screen0" 0 0
    Screen      1  "Screen1" below "Screen0"
    Screen      2  "Screen2" below "Screen1"
    Screen      3  "Screen3" below "Screen2"
    InputDevice    "Keyboard0" "CoreKeyboard"
    InputDevice    "Mouse0" "CorePointer"
    Option         "Xinerama" "1"
    Option         "StandbyTime" "0"
    Option         "SuspendTime" "0"
    Option         "OffTime" "0"    
    Option         "BlankTime" "0"
EndSection

Section "Files"
EndSection

Section "InputDevice"

    # generated from default
    Identifier     "Mouse0"
    Driver         "mouse"
    Option         "Protocol" "auto"
    Option         "Device" "/dev/psaux"
    Option         "Emulate3Buttons" "no"
    Option         "ZAxisMapping" "4 5"
EndSection

Section "InputDevice"

    # generated from default
    Identifier     "Keyboard0"
    Driver         "kbd"
EndSection

Section "Monitor"
    Identifier     "Monitor0"
    VendorName     "Unknown"
    ModelName      "Philips 200P"
    HorizSync       30.0 - 97.0
    VertRefresh     56.0 - 85.0
    Option         "DPMS" "false"
EndSection

Section "Monitor"
    Identifier     "Monitor1"
    VendorName     "Unknown"
    ModelName      "Philips 200P"
    HorizSync       30.0 - 97.0
    VertRefresh     56.0 - 85.0
    Option         "DPMS" "false"
EndSection

Section "Monitor"
    Identifier     "Monitor2"
    VendorName     "Unknown"
    ModelName      "Philips 200P"
    HorizSync       30.0 - 97.0
    VertRefresh     56.0 - 85.0
    Option         "DPMS" "false"
EndSection

Section "Monitor"
    Identifier     "Monitor3"
    VendorName     "Unknown"
    ModelName      "Philips 200P"
    HorizSync       30.0 - 97.0
    VertRefresh     56.0 - 85.0
    Option         "DPMS" "false"
EndSection

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "Quadro NVS 290"
    BusID          "PCI:1:0:0"
    Screen         0
EndSection

Section "Device"
    Identifier     "Device1"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "Quadro NVS 290"
    BusID          "PCI:1:0:0"
    Screen         1
EndSection

Section "Device"
    Identifier     "Device2"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "Quadro NVS 290"
    BusID          "PCI:2:0:0"
    Screen         0
EndSection

Section "Device"
    Identifier     "Device3"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "Quadro NVS 290"
    BusID          "PCI:2:0:0"
    Screen         1
EndSection

Section "Screen"

    Identifier     "Screen0"
    Device         "Device0"
    Monitor        "Monitor0"
    DefaultDepth    24
    Option         "Stereo" "0"
    Option         "nvidiaXineramaInfoOrder" "CRT-0"
    Option         "metamodes" "DVI-I-0: nvidia-auto-select +0+0"
    Option         "SLI" "Off"
    Option         "MultiGPU" "Off"
    Option         "BaseMosaic" "off"
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

Section "Screen"

    Identifier     "Screen1"
    Device         "Device1"
    Monitor        "Monitor1"
    DefaultDepth    24
    Option         "Stereo" "0"
    Option         "nvidiaXineramaInfoOrder" "CRT-1"
    Option         "metamodes" "DVI-I-1: nvidia-auto-select +0+0"
    Option         "SLI" "Off"
    Option         "MultiGPU" "Off"
    Option         "BaseMosaic" "off"
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

Section "Screen"

    Identifier     "Screen2"
    Device         "Device2"
    Monitor        "Monitor2"
    DefaultDepth    24
    Option         "Stereo" "0"
    Option         "nvidiaXineramaInfoOrder" "CRT-0"
    Option         "metamodes" "DVI-I-0: nvidia-auto-select +0+0"
    Option         "SLI" "Off"
    Option         "MultiGPU" "Off"
    Option         "BaseMosaic" "off"
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

Section "Screen"

    Identifier     "Screen3"
    Device         "Device3"
    Monitor        "Monitor3"
    DefaultDepth    24
    Option         "Stereo" "0"
    Option         "nvidiaXineramaInfoOrder" "CRT-1"
    Option         "metamodes" "DVI-I-1: nvidia-auto-select +0+0"
    Option         "SLI" "Off"
    Option         "MultiGPU" "Off"
    Option         "BaseMosaic" "off"
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

So, that’s what I got. Exciting, right? I hit a bunch of snags while trying to get 4 monitors working, so I figured I would share my findings.

  • The open-source nouveau driver could not keep up with all 4 monitors, so I had to install the nvidia drivers. I installed the nvidia-352 package (apt-get install nvidia-352) and things seem to be working fine.
  • Unity doesn’t seem to play nice on the video card, so installing Gnome 3 was a necessity. (apt-get install gnome-desktop-environment)
  • The nvidia-settings tool and twinview proved difficult, so I used 4 xscreens and xinerama.
  • So, there’s not a lot of documentation on getting 4 monitors working in Xinerama in xorg.conf, most multi-monitor setups are 2, or at most 3. To support 4 monitors on 2 video cards, I had to duplicate the devices and give them unique “Screen” identifiers. BUT NOTE! Each device starts at Screen 0, it’s not a global setting.
  • On the flip side, in ServerLayout, the Screen fields are numbered from 0-3 when setting position. These don’t directly map to the previously defined Screen fields. (I was confused for a few hours on this point!)
  • Disabling the screensaver and the power settings/standby timer did nothing for stopping the screen timeout, so I had to configure the monitors to not report timeouts. Each Monitor has the DPMS option set to false, and the ServerLayout has a bunch of settings to disable all kinds of timeouts.

After a few days of trial-and-error (and ssh-ing in to reset xorg.conf) I was able to get a quad-monitor, always-on info center working in the office.

IMG_20160222_134821

Ubuntu 15.10 Quad Monitor xorg.conf configuration

What is a Sustainable Developer?

I was recently asked to describe what I felt were the traits that exemplified modern, professional software engineers. I wrote pages of notes trying to describe how I felt and why, but the most effective way I found to describe what a Sustainable Developer does is to point out the differences from the traditional (i.e. bad) way of developing code.

Traditional development practices

  • new features are done as quickly as possible
  • changes are done as quickly as possible
  • each developer works in a single part of the code
  • developers rarely talk with each other except to decide on an API
  • bugs are an inherent part of having custom software
  • servers and software will randomly break and we work all night to get it back up
  • the tech team’s work is too volatile to do meaningful long-term planning
  • bigger changes mean bigger risks
  • eventually the code is rewritten because the existing codebase is unmanageable

Sustainable development practices

  • Developers are a team and everyone is responsible for every part of the code
  • Developers follow the Boy Scout Method: if you touch code, you must leave it better than when you found it, never worse
  • Developers use emergent design and the SOLID principles to guide how code is written
  • The developers will work closely with the customer to deliver exactly the feature required
  • New features and changes are done in order of business priority
  • Happy developers leads to good decisions – deadline pressures are detrimental to solving a problem effectively
  • Code quality is never something that can be compromised; only features, time or resources are the dials to change
  • Developers don’t write bugs in the first place so they never have to debug later
  • Software is trustworthy; changes are never scary because the code is well encapsulated, easy to read and never “clever”
  • Code will have automated unit tests over it to alert other programmers when they might be introducting an unexpected side effect
  • servers are stable and trustworthy; instability and failures are rarities
  • Development teams are never larger than 5-10 people; developers cannot communicate effectively to a large group
  • Pair-programming is a must when training new developers in the codebase or when sharing knowledge between senior and junior developers
  • Team health and current status are always clearly communicated – less “unexpected” surprises and planning is a meaningful activity

Summary

So really, this boils down to about 6 major viewpoints that differentiate a sustainable developer from a traditional one.

Stability and quality are the primary focus of Sustainable developers

Having bugs in the code is comparable to having cracks in a housing foundation; each one is dangerous and expensive to fix. Nothing unexpected should ever happen when using or changing the software.

Sustainable Developers maintain codebases for the long-term

We don’t work in churn-and-burn projects; we live in this codebase now, we better prevent an infestation of bugs. The code needs to be easy to read, easy to maintain and even easier to debug. Code that does something wrong should be very easy to spot because the code is simple, clear and concise.

Communication between the developers and the customer drives all code work

We only build what is requested, and we build exactly what is requested. Requirements gathering is a difficult part of the job, but prevents us from wasting time on incorrect or useless features.

Communication within the development team

We all have strengths and weaknesses, learning and cooperation is best for the team. Daily standups so the team is aware what others are working on, weekly or monthly retrospectives to analyze longer-term issues or to air grievances within the team are all important for keeping the developers happy and the code running smoothly.

Education and experimentation

Well-informed developers make well-informed decisions; lectures, classes, building prototypes, and research are all crucial parts of a developer’s day-to-day.

Time pressures are kept away from developers whenever possible

Effective planning is critical to keep developers from the “death march” typically associated with traditional development. Late nights and sleep deprivation only lead to bad coding decisions and bugs in the future. If a developer is informed that a feature must be out by a certain date, the temptation to cut corners to try and hit that date is large, but the long-term cost is much larger.

What is a Sustainable Developer?

Unreal Editor 4 for Linux

I recently discovered that the Unreal Editor 4 has an early-access branch available for Linux! So I decided to pull it down and give it a whirl in Ubuntu 15.04.

So if you want to use the free/open-source version of the editor, first you have to request access to the github repository. You’ll need to make an Epic Games account and once you’re logged in, link your github username to your account from the Profile tab. These instructions can be found here: https://www.unrealengine.com/ue4-on-github

Once you have access to the repo, make sure you’re on a beefy computer, since we’ll be compiling massive amounts of C++ code.

Update: Ugh, ok, so this was even more intense to compile than I expected. So I could continue using my computer while this runs, I decided to add some niceness to the compilation process. This will slow down the build process, but it won’t murder your computer if you need to use it, either.

git clone -b 4.8 https://github.com/EpicGames/UnrealEngine.git
cd UnrealEngine
./Setup.sh
./GenerateProjectFiles.sh
nice -n20 make SlateViewer
nice -n20 make UE4Editor ARGS=-clean
nice -n20 make UE4Editor UE4Game UnrealPak CrashReportClient ShaderCompileWorker UnrealLightmass
cd Engine/Binaries/Linux && ./UE4Editor

Only 2 hours later, I finally have it working!
unreal_editor_in_linux_desktop

I’ve got several thousand shaders to compile before I can actually get into using it, but it looks like the editor is fully featured and working pretty well.

Unreal Editor 4 for Linux

Ubuntu 15.04 Audio Output Keyboard Shortcut

My computer is located in my bedroom, which is awesomely convenient except when I want to play audio. Usually it’s not a problem to play it out loud, but in the evening it’s gotta be headphones or muted. Ubuntu makes it easy to change audio sources using the Sound Manager in the navbar, but change sources 3-4 times a day and it starts getting annoying quick.

audio_selector

I’m surprised that there is no easy way to switch sources, so I decided to set up a custom keyboard shortcut to run a script that toggles between sources.

First, you’ll need the script.

#!/bin/bash

declare -i active_sink=`pacmd list-sinks | sed -n 's/.*\*.*index: \([0-9]\)/\1/p'`
 declare -i next_sink_index=0
 declare -i available_sinks=(`pacmd list-sinks | sed -n 's/.*index: \([0-9]\)/\1/p'`)
 for index in ${!available_sinks[@]};
 do
 if [ ${available_sinks[$index]} -eq $active_sink ]; then
 next_sink_index=${index}+1
 fi
 done
 if [ $next_sink_index -ge ${#available_sinks[@]} ]; then
 next_sink_index=0
 fi
 new_sink=${available_sinks[$next_sink_index]}
 pacmd "set-default-sink ${new_sink}"
 declare -i apps_playing=`pacmd list-sink-inputs | sed -n 's/.*index: \([0-9]\)/\1/p'`

 for app in $apps_playing;
 do
 pacmd "move-sink-input $app $new_sink"
 done

Essentially, this script uses pacmd list-sinks and pacmd list-sinks-inputs to scrape out information about the currently available and playing audio. It builds a list of valid outputs, and then it will flip all playing sources through all available outputs.

Place the script wherever you’d like, make it executable and then set up the keyboard shortcut.

System Settings > Keyboard > Shortcuts > Custom Shortcuts > +

shortcut_define

Name the shortcut something memorable, and then set the command to point to your sound-switcher script. Mine is /home/midas/sound-switcher.sh

After clicking Apply, you just need to bind it to a hotkey or combination.

keyboard_shortcuts

I picked F9 because my keyboard has a little music note there, so it makes the most sense for me.

Hopefully this makes your audio-flipping life a little easier. Let me know if you come across any problems, and I’ll be happy to help.

Ubuntu 15.04 Audio Output Keyboard Shortcut

nginx CORS whitelist map

Above your server {} block, you need to add a whitelist map.

map $http_origin $cors_header{
  default "";
  include /etc/nginx/cors-whitelist.map;
}

Then you need to create a cors-whitelist.map file in /etc/nginx.

"~^(http:\/\/localdomain1\.com)$" "$http_origin";
"~^(http:\/\/localdomain3\.com)$" "$http_origin";

And finally, add in the appropriate headers in your server block:

server {
  listen 80;
  listen [::]:80;

  add_header 'Access-Control-Allow-Origin' "$cors_header";
  add_header 'Access-Control-Allow-Credentials' 'true';
  add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';

root /var/www/html;
[...]

It’s a little redundant in the map file, but it maximizes readability if you have a lot of servers that need CORS access.

nginx CORS whitelist map

How to record desktop images into gif format on Ubuntu 14.04

First, you need to install a few requirements:
sudo apt-get install gifsicle imagemagick
sudo add-apt-repository ppa:maarten-baert/simplescreenrecorder
sudo apt-get update
sudo apt-get install simplescreenrecorder

Now you need to use simple screen recorder to capture whatever video you like. I used it to snag some gameplay of a game I was working on. It has an excellent GUI, and I drew a rectangle on the screen around the game I wanted to record, and saved it.

Now that you have a video, it’s time to convert it into an exciting gif! First, use avconv to break the mp4 into it’s component frames. The stuff in brackets is optional.

avconv -i lights_gameplay.mp4 -vsync 1 [-r 25] -an -y -qscale 1 [-s 800x600] lights/out_%04d.png

Woo! This is a big confusing command and is difficult to just discover what it’s doing. So here’s a breakdown!

  • -i [filename] this is the input filename
  • -an Ignore the audio
  • -y allow overwrites without asking
  • -r [fps] frames per second, if you want to set it to something different than your video
  • -qscale [amount] 1-31, with 1 being the best and 31 being awful
  • -vsync [method] I have no idea what this does. Just set it to 1.
  • -s [output resolution] If you want to start with a particular image scale.

I used this because my video was already perfect.
avconv -i lights_gameplay.mp4 -vsync 1 -an -y -qscale 1 lights/out_%04d.png

Note: I didn’t convert directly to gif here because you lose the color information. Since I didn’t select *exactly* the game window, I used an image editing program (gimp) to figure out how many pixels I wanted to crop off of the edges.

I want to remove 14 pixels from the left side, and 12 pixels from the top, so that my remaining image is 800×600 (the size of my game screen).

for i in lights/*.png ; do convert "$i" -crop 800x600+14+12 +repage "$i" ; done

At this point, there was a bit of “video editing” I wanted to do. I just opened the folder with the pngs and deleted out the frames I wanted to remove. To reduce the file size, I also decided to scale the images down. This is not required, but if your gif ends up massive, this is the first recommendation I would make for improving it.

for i in lights/*.png ; do convert "$i" -resize 50% "$i" ; done

Now that our frames are just what we want, let’s finally turn them into gifs! (This takes a long time if you have a bunch of frames, so just be patient!)
for i in lights/*.png ; do convert "$i" "${i%.*}.gif" ; done

Gifsicle is an animated gif manipulation tool. We’re using it just to piece together an animated gif from the frames, but it allows us to change some standard gif parameters in the process. For instance, this sets the delay between frames to 3, and sets the gif to loop forever.
gifsicle --delay=3 --loop lights/*.gif > lights_raw.gif

Finally, try to have imagemagick strip out extraneous data (bits of each frame that don’t change)
convert -layers Optimize lights_raw.gif lights_gameplay.gif

or, if you want to be really aggressive in compressing the gif (but you run the risk of artifacts)
convert lights_raw.gif -fuzz 30% -layers Optimize lights_gameplay.gif

And here’s the finished product! Only a tiny 7.9 mb!

lights_gameplay

How to record desktop images into gif format on Ubuntu 14.04