Category: /dev/random

Entropy lacking write stream

Building Python 3.5 for CentOS

Update: As per Alexander Todorov’s suggestion using https://www.softwarecollections.org/ is much easier and saves a lot of trouble compiling.

So after git it’s time to upgrade python. CentOS 7 has python 3.4, but there are some small but very annoying incompatibilities with 3.5. And it seems that 3.5 will be the major python 3 release. So here we go:

RPM build chroot

First lets create the chroot directory and  initialize the  rpm database   :

Next  install the latest centos-release package,  yum and add the EPEL inside the chroot:

Next we continue inside the chroot :

Building RPMs

Next we need to install gcc, and few other packages that are needed to compile RPMs and pyton.

Pick a python 3.5 SRPM from https://apps.fedoraproject.org/packages/python3 .

And install it :

You can just ignore the warning about missing users. Go to SPECS directory and edit python3.spec so that :

becomes

This has something to do with a circular dependency of the rewheel package. Now we can build python with:

After the build is done we can check if the packages are installable:

Now let’s try to rebuild with the rewheel package. We will need python-rpm-macros, python3-pip and python3-setuptools  and a bunch of other packages for building them 🙂

Change back python3.spec to:

And now rebuild python3 again:

After the build finishes we have the RPMs in ~/rpmbuild/RPMS:

Exit the chroot and install the newly build python 3.5 packages :

Building latest git for CentOS

CentOS is a great operating system, but recent versions of software packages is not one of its virtues. For example the current CentOS 7.2 ships with git version 1.8.3 released 2013, almost 3 years ago. And by that time git gained quite some useful features. So lets see how to build RPMs with latest git.

RPM build chroot

First thing is to make a chroot where to build the RPM, because there are a lot of dependencies required for building, which needs to be cleaned up afterwards. Much easier is to just delete the whole chroot dir 🙂

First lets create the chroot directory and  initialize the  rpm database   :

Next  install the latest centos-release package and yum inside :

Next we continue inside the chroot. Lucky us, systemd-nspawn makes it trivial to work inside the chroot :

 

Building RPMs

Next we need to install gcc, and few other packages that are needed to compile RPMs .

And now we need a git SRPM to build. We can just grab the Fedora one which has git 2.7.1. Here https://apps.fedoraproject.org/packages/git/overview/ are all Fedora builds, and I picked the Rawhide version. Clicking on it takes us to a page with build information and there is also a link to download the SRPM for this build. Lets install it :

You can just ignore the warning about missing users. Go to SPECS directory and try to build the RPMs:

Ok, we need to install some dependencies to be able to build. Lets just install them:

and run rpmbuild again:

After the build finishes we have the RPMs in ~/rpmbuild/RPMS:

Exit the chroot and install the following packages for a minimal git installation:

Not, so bad 🙂 Next time I will check how to create a repository .

Quck and (not so) dirty compressing reverse proxy.

While working on a couchapp recently I found out quite interesting fact. CouchDB doesn’t support   the gzip/deflate HTTP responses. And with a view that’s several MB, on a slow connection it was a lot pain using the app.

My first thought was, no problem, my CouchDB is behind nginx anyway, lets just turn on gzip compression in nginx. And while that was super simple to do, it yielded an undesired effect. The responses were now compressed, but nginx strips off Etags headers, and there is no way around it. Without the Etag queries always return full responses, even if the data hasn’t been modified. With them, a short 304 reponse is sent when there is no change.

Unhappy with nginx approach to way too strict HTTP, I decided to write my own compressing proxy. Fortunately thats super simple with nodejs/iojs. Its just a matter of gluing few modules together 🙂

First install the modules:

Then save this as proxy.js in the same directory:

And voila , now run:

And you have gzipped responses from CouchDB on port 8012.

 

Unfrozen, or how to get that file back from Eclipse

I was happily working on a small java project, and after just creating a new Java class, and typed around 100 lines in the Eclipse editor, the whole IDE  froze.  It seems there is a bug in Luna that will freeze the IDE on out of memory condition, and I have just hit it.

100 lines is not that much of a loss generally, I can write them again. But why? Aren’t computers supposed to save us from the manual work. I wanted this file back 🙂 Quick check on the file system showed that after creating the class Eclipse saved and empty class inside. I had to find another way I guess.

What if I somehow manage to connect a Java debugger and search for the file in memory. Unfortunately I am not that proficient in Java and  doing all the research on how to do it may take quite some time.  So lets see do I really need a debugger ? I just need the file, and it is somewhere in the Eclipse process memory. Getting the whole memory of a process is quite easy in Linux :

And voila, we have the whole memory of the process saved to core.<PID> . Now lets find that file. strings comes to mind instantly :

And I started searching for occurrence of ‘class MetadataInfo’ ( that was my class name) . Unfortunately while  MetadataInfo had plenty of instances none of them was preceded by a class keyword and none was actually in the content of my file.  Hmm, maybe the file isn’t stored as plain string, maybe it is some kind of tree for easier syntax t highlighting, as there were pieces of the file all over the place. But then it occurred to  me that the file is probably stored in UTF16 in memory as Java is very much into UTF16. Quick check shows that strings supports several encodings :

Nice! Lets try 16-bit LE  as I am on an Intel machine:

Now searching  for ‘class MetadataInfo’ finds several occurrences of the full file, I guess because of the Local History that Eclipse keeps for each file.  Just copied one that seems good enough, saved it to file and killed the frozen Eclipse w/o significant data loss.

 

Btrfs defragment

I have BTRFS for root and home volumes on my latop, and today I found out that  BTRFS supports defragmentation. It’s interesting to check whether defragmentation has measurable performance impact, so I did a quick test.

The easiest thing was to measure the boot time as reported by systemd .  So I did a couple of restarts, recorded the boot times systemd-analyze reports :

Before defragment Kernel initrd user Total
Boot 1 2.743 2.015 4.81 9.568
Boot 2 2.749 1.913 4.757 9.419
Boot 3 2.79 1.882 3.209 7.881
Avegare 2.761 1.937 4.259 8.956

Then I did a :

And recorded the new boot times :

After defragment Kernel initrd user Total
Boot 1 2.751 1.734 3.37 7.855
Boot 2 2.743 1.76 4.287 8.79
Boot 3 2.746 1.747 2.847 7.34
Average 2.747 1.747 3.501 7.995

That’s a whole second less, or about 11% improvement.  Not so bad at all, given that the disk is Samsung 840 Pro SSD, and is quite fast anyway.

tmux on Solaris

In the last month my work is somewhat related to Solaris and since I really got used to tmux I kind of miss it. Unfortunately it turns out Solaris is not the happy Linux land I am used to. One does not simply  dowload, ./configure and make install stuff in Solaris. Not without   the proper spells 🙂 So here is an attempt to document my experience to build static tmux binary, that doesn’t require libevent or ncurses shared libraries. I used a VM I had with gcc installed from openCSW.

First thing libevent, download and unpack it. Then:

Next goes ncurses,  download  and unpack. Then:

And finally we can compile tmux itself. This is abit trickier , as it seems tmux doesn’t directly support Solaris. Download and unpack.  You need to apply the following patch:

Use gpatch, obviuosly patch in Solaris carries some braindead Unix legacy which I don’t even want to know about:

Then we setup the CPP/LDFLAGS, ./configure and make:

Thats it 🙂 In the tmux source directory we have a tmux binary that depends only on system libraries so you can copy it to other Solaris boxes and use it there.