How to check number of cores per CPU

echo $(($((`cat /proc/cpuinfo | grep 'processor' | sort -n | wc -l`))/$((`cat /proc/cpuinfo | grep "physical id" | sort -n | uniq | wc -l`))))

How to do a platform-independent path join in Java

Today I had to join a file path with its parent directory path. I could this simply by joining strings with "/", but this solution is not portable to non *nix systems. In Python I would write:

import os.path

parent = "a/b/c"
filename = "efg"
joined = os.path.join(parent, filename)

But what is Java equivalent? It turns out that the best way is to use java.io.File. One can do:

File parent = new File("a/b/c");
File file = new File("efg");
String joined = new File(parent, file).getPath();

The last line is to go back to String world, but possibly you don't have to.

Why we all love Maven

Synchronized print in Python

Today I faced a problem where I had to print to the standard output in a synchronized way. I put the following code in a file named rpython.py

from __future__ import print_function
from threading import Lock

_global_lock = Lock()
_old_print = print 

def print(*a, **b):
        with _global_lock:
                _old_print(*a, **b)

Now, in every Python module that requires synchronized printing I typed:

from __future__ import print_function
from rpython import print

Alternatively, if you don't want to shadow the default printing you can type:

from __future__ import print_function
from rpython import print as rprint

Have a nice, synchronized day.

EDIT: I should have started with a clear statement that Python's print is not synchronized in any way. Multiple threads writing concurrently to any stream can interfere and usually they do that, so as an effect you get a random mixture of massages which looks really ugly.

util.NativeCodeLoader: Unable to load native-hadoop library for your platform.

Problem: when launching a hadoop-based app one can encounter message

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... 
using builtin-java classes where applicable

Solution: type following command in the command line. The path may vary depending on your platform and hadoop version

export LD_LIBRARY_PATH=/usr/lib/hadoop-0.20-mapreduce/lib/native/Linux-amd64-64

How to build jar with Maven containing nothing but dependencies

Today I came across a problem where I had to split my project into two jars: one containing my project's classes and the other one with dependencies. The former got changed every time I made some changes in the source files. The latter was basically constant throughout the development. This facilitates me the deployment of my application. Previously after each improvement I had to transfer a big-fat-and-ugly *-jar-with-dependencies.jar and that took ten good minutes as the file weights 189MB. It turned out that dependencies weight.. 189MB and my compiled classes 45KB. Below the xml that I placed in src/main/resources/assemblies/only-deps.xml

  
  only-deps
  
    jar
  
  false
  
    
      /
      false
      true
      runtime
    
  

and a profile that I added to the main pom.xml:

        
            only-deps
            
                
                    
                        org.apache.maven.plugins
                        maven-assembly-plugin
                        
                            
                                src/main/resources/assemblies/only-deps.xml
                            
                        
                        
                            
                                package
                                create-my-bundle
                                
                                    single
                                
                            
                        
                    
                
            
        

Run the whole thing with mvn install -P only-deps.

How to make Windows executable from a Python script

Recently I faced a problem that I always wanted to stay away from. My brother asked me to write a simple GUI app, which isn't a big deal, but it has to be run in Windows. Holy shit. I develop all the staff in Linux and suddenly I have to prepare a Windows-compatible app. As Python is a language of my choice, I picked it also this time. I wrote an app under Linux and then I made a research on how to port it to Windows without a need to install Python on the target machine. I found cx_Freeze which seemed to be suitable. Having read a really good cx_Freeze tutorial I wrote a setup.py script which looked like this:

#!/usr/bin/env python
from cx_Freeze import setup, Executable

exe = Executable(
    script="google_check.pyw",
    base="Win32GUI",
    )

setup(
    name = "googleCheck",
    version = "0.1",
    description = "google check app",
    executables = [exe]
    )

then run

pawel@pawel-asus:/media/5841-E712/seba$ python ./setup.py build
running build
running build_exe
Traceback (most recent call last):
  File "./setup.py", line 13, in 
    executables = [exe]
  File "/usr/lib/pymodules/python2.7/cx_Freeze/dist.py", line 278, in setup
    distutils.core.setup(**attrs)
...
  File "/usr/lib/pymodules/python2.7/cx_Freeze/freezer.py", line 186, in _GetBaseFileName
    raise ConfigError("no base named %s", name)
cx_Freeze.freezer.ConfigError: no base named Win32GUI

Uhhh.. I can't freeze an app for Windows under Linux. My bad. There is no way to "cross-compile" an app on Linux without having a Python installation on Windows partition that would be visible from Linux. If you have one, you can use PyInstaller. I had no Python on my Windows so I had to install it first.. which is a pain in the ass. Here is what to do:

1. Go to Python for Windows download page, download an installation file and install it.
2. Add your Python/scripts path to the PATH variable (go to My computer/properties/Advanced/Environmental Variables/Path/Edit then paste the path with an extra semicolon as the last entry.
3. Get Easy install installer and install it.
4. Open command line and type easy_install cx_Freeze

Now you are ready to go. Change the working directory to where you placed your application and setup.py, type cxfreeze ./setupy.py and look if there are some modules missing. If it is so, install them with easy_install. It may happen, that you won't be able to get some module in this way. Well, cross your fingers, in the end they maybe unnecessary at all to run your project. Now go to a newly-created dist folder. There is your double-clickable python application. Bon apetit.