Working with ZIP in terminal

Couple of days ago wanted to use ZIP in OSX terminal and realised that I don't have examples really (yes U can check manuals etc.) so I've decided just to note most common / usable ones:

- zip --encrypt file.zip files - will create an encrypted zip archive (files won't be removed automatically after compressing);

- zip -r archive.zip directoryName - will compress whole directory at once. U can add --encrypt flag here as well

- unzip file.zip - extracts file from archive into current directory

That's it for now although most likely will have to add more later on.

Working with files in Python

Some notes about working with files in Python.

How simply generate path:

import os
os.path.join('usr', 'user', 'Documents')

or if we need path to file:

os.path.join('home/Documents/work', important.txt)

os.getcwd() - check current directory

os.chdir('path to new working directory') - change directory

os.makedirs('home/docs/test') - create directory

os.path.abspath('path') - get absolute path

os.path.relpath('path', 'start point') - get relative path

os.path.dirname('path') - get directory name actually will return name with whole path

os.path.basename('path') - get file name

os.path.split('path') - to get both directory name and file name as a tuple

or 

examplePath.split(os.path.sep) - will return tuple where every directory and file name are separate strings (in OSX first item in the list will be '')

os.path.listdir('path') - list all items in the directory

os.path.getsize('path') - get file size in bytes

os.path.exists('path') - returns True if directory or file exists

os.path.isfile('path') - returns True if file exists

os.path.isdir('path') - returns True if directory exists

open('path') - returns file object if there's no such file Python will create that

read('path') or open('path', 'r')- reads file

file.readlines() - reads line by line from file

write('path') or open('path', 'w')- writes to file (simple usage will overwrite all content)

open('path', 'a') - will append text to the end of the file instead of overwriting

close('path') - closes file


Work with binary shelf files (can be used as dictionaries etc.):

import shelve
testFile = shelve.open('filename')
testValues = ['email', 'address', 'phone']
testFile['data'] = testValues
testFile.close()

testFile('data') - prints out values

list(testFile.keys()) - prints list of keys in our case it will be just 'data'

list(testFile.values()) - prints list of values ie ['email', 'address', 'phone']

import pprint
pprint.pformat(list) - will represent list / dictionary as a string for easier saving that to a file

And for the end short reminder about shelf files and plain text files: first used mostly for complex objects (ie file objects) while plain text files can be used for storing simple data (ie text, integers etc)

Math calculation with Bash in OSX

There are quite a few options how to calculate with Bash in OSX terminal but for reference I'd like to mention just several of them which can be used as a baseline.

echo "2 * 2" | bc

echo "scale=3; 1000/52 | bc" - will output result + 3 digits after comma

bc <<< "100 / 3"

bc -l <<< "sqrt(5)" (scale(3.3) - number of digits after comma, length(100) - number of digits)

awk "BEGIN {print 33 * 33}"

Change File Permissions in Terminal OSX

Knowledge about changing permissions for a specific file is pretty important and can even save Your information from being lost.

So let's see what we can check and how.

1. Execute next command in Your terminal window:

$ ls -la

You should see same information as on screen below.

What do those letters form first column stand for?
- d = directory
- r = read access
- w = write access
- x = execute access
- - = no permissions at all

Column 3 indicates owner of a file and column four shows us an user's group which might has or not permissions to read/write/execute file.

So if we'll check first column again from left to write we'll see that:

- first symbol / several symbols - user's permissions
- second symbol (after dash) / symbols - group permissions
- third symbol / symbols - all other users permissions

Let's analyse example.txt file:

- user has read and write permissions
- group has read and write permissions
- other users have only read permission

2. If You want to change ownership of the file just execute:

$ sudo chown NEW_USER:NEW_GROUP example.txt

this command will update owner and group for this file.

3. If You want just update permissions for specific file then You need to execute:

$ sudo chmod 000 example.txt

this command deny all actions with this file for all users.

4. Let's explore octets in general:

- 0 - No read, no write, no execute  - " - "
- 1 - No read, no write, execute       - " -x "
- 2 - No read, write, no execute       - " -w- "
- 3 - No read, write, execute            - " -wx "
- 4 - Read, no write, no execute      - " r- "
- 5 - Read, no write, execute           - " r-x "
- 6 - Read, write, no execute           - " rw- "
- 7 - Read, write, execute                - " rwx "

5. So according to this table if we'll execute next command:

$ sudo chmod 432 example.txt

- owner will be able only to read file;
- group will be able to write and execute;
- all others will be able only to write.

6. If You want to apply same permission to folder and all its content You have to add one more flag to Your command:

$ sudo chmod -R 777 Documents

This is pretty much core knowledge which You have to have in order to change permissions for folder / file in OSX terminal.

Use Multiple Python Versions In OSX

This post aims to help with managing several different Python versions on OSX without any troubles.

As usually just follow next steps:

1. $ brew update

2. $ brew install pyenv

3. Add into .bashrc: 

" if which pyenv > /dev/null; then eval "$(pyenv init -)"; fi"

4. Shows list of options

$ pyenv install -l  

5. Installs needed python version

$ pyenv install X.XX.XX 

6. Shows installed python versions in system

$ pyenv versions 

7. Sets specific version as global one

$ pyenv global X.XX.XX 

8. Sets specific version as local one

$ pyenv local X.XX.XX 

9. Unsets local version for python

$ pyenv local --unset 

For further exploring:

10. Shows list of available commands for pyenv

$ pyenv commands 

This is pretty much enough for this topic (at least for now).

Use Multiple Java Versions In OSX

Really common question today is - "How I can install and use several Java versions on my OSX without a headache?", so here's my answer.

Just follow simple and straightforward steps below (yes I did check those before publishing):

1. Check if You already have java installed:

$ java -version

2. If You have java locally then check location of installed version:

- installed from Apple - 

/System/Library/Java/JavaVirtualMachines/1.X.X.jdk/Contents/Home/

- installed from Oracle - 

/Library/Java/JavaVirtualMachines/jdk1.X.X_XX.jdk/Contents/Home

Remember that path.

3. Check if You have homebrew installed if not install it first:

- http://brew.sh/

4. Install jenv with homebrew:

$ brew install jenv

5. Add information about jenv into Your .bash_profile or .bashrc file:

if which jenv > /dev/null; then eval "$(jenv init -)"; fi

6. Save and reload .bashrc / .bash_profile:

$ source .bashrc (.bash_profile)

7. Check that jenv works properly:

$ jenv versions

This command should have next output by default:

$ * system (set by /Users/ajones/.jenv/version)

8. Add Your installed local java to jenv:

$ sudo jenv add /Library/Java/JavaVirtualMachines/jdkX.X.X_XX.jdk/Contents/Home/

Check if this added with jenv versions command.

9. Install cask with homebrew:

$ brew install cask

10. Install latest java from Oracle:

$ brew cask install java

Currently (March 2016) this command installs jdk1.8.0_74.jdk from Oracle.

11. Add latest installed java into jenv as peviously:

$ sudo jenv add /Library/Java/JavaVirtualMachines/jdkX.X.X_XX.jdk/Contents/Home/

12. Run jenv versions and verify that now You have 2 java versions installed on Your local machine

13. If You want to switch to needed version run:

$ sudo jenv local oracle64-1.7.X.XX or
$ sudo jenv local oracle64-1.8.X.XX

14. After each command You can run:

$ java -version

in order to check which java version is currently active.

15. If You want to change java version to all users:

$ sudo jenv global oracle64-1.7.X.XX or
$ sudo jenv global oracle64-1.8.X.XX


You can install as many java versions as You want - just add them to the jenv and then simply do switch when needed.

BASH cheat sheet

Traditionally I'll put here some well known BASH commands for keeping those in one place in case I'll forget them :)

echo $BASH - should be /bin/bash - make sure that U're using bash :)

nano (vim) ~/.bash_history - shows all history of commands in bash
reset - resets terminal after crashing (eg trying to output binary file with less command etc.)

ls - list of content in current directory
ls -a - list of content in current directory including hidden files
ls -l - list of files in long format
ls -a /bin - list of all files in bin directory
ls -a Documents/ - list of all files in Documents directory without entering into that
ls -t - list by modification date
ls -S - list by size
ls -R - recursively list files in subdirectories
ls -d - don't go into subdirectories just list them
ls | wc -l - show how many files in the current directory

man - help for comands (Manual pages)
man ls - shows help related to ls command

Navigation inside man:
space - scrolls page down
b - scrolls page up
/ <option> - performs search inside help; for next occurrence of searched term press / + enter
q - exits help

pwd - Path to working directory
cd - changing directory / if without arguments navigates to Home directory
cd / - navigates to Root directory

file <filename> - shows information about file without opening it
file * - shows information about all files in current directory
open <filename> - opens file with default (specified in system) application
open <filename> -a <application> - opens file with wanted application
open . - opens current directory in Finder
cat <filename> - print content of file into console
less <filename> - show content of file with possibility of navigation (like inside of man)

tail <filename> - show end of the file
tail <filename> -n 5 - show last five lines of file
tail <filename> -f - show latest (realtime) updates in the file, suitable for debugging
head <filename> - show beginning of the file

mkdir <name> - creates a directory
touch <filename> - create an empty file (updates modification and access dates in case file already exists)
touch {a,b,c,}.txt ==> touch a.txt b.txt c.txt
touch {a..c}{1..3}.txt ==> touch a1.txt a2.txt ... c2.txt c3.txt

cp <filename with relative path> . - copy file to current directory
cp <filename with relative path> <new file name> - copy file to current directory with different name
cp -R <source directory / file> <target directory> - copy source directory / directories / files into target directory
rm <filename> - removes a file
rm -r <directory> - removes a whole directory
rmdir <directory> - removes an empty directory
mv <filename> - move a file (also can be used during renaming)
mv <filename>.. <directory> - move file / files into specified directory
mv file.{txt,jpg} dir/ ==> mv file.txt file.jpg dir/
cp / mv / rm -i - security switch - will ask if you are sure to perform an action

ls > <filename> - write content of directory into file - will override file  if exists
ls >> <filename> - write content of directory into file - will append information at the end of the existing file, otherwise will create a new file

ls | less - get output from first command and send it to input of second command

cut -f 3 <filename> | grep 4 | wc -l - get 3 rd column from file (assuming it's csv) then get all rows which contains 4  and then count how many lines contains that number

CTRL-r - reverse search in bash history - search for the latest occurrence. If wee need previous one just press CTRL-r again and so on. To exit from search mode without execution command we can use arrow (left, right) key

sort -nk2 <filename> - sort file by using second column (in space-separated file) as a key
sort -r <filename> - sort in reversed order
sort <filename> | uniq -c - sort file then counts how many time each line appeared
wc <filename> - show number of lines, words and bytes of the file
wc -l <filename> - show opnly number of lines

grep <search term> <filename> - search for all occurences in the file
grep <search term> *<filename pattern> - search for all occurences in several files
grep -i <search term> <filename> - search for all occurences in the file ignoring case
grep -v <search term> <filename> - invert search

find /bin - search for all files in bin directory
find /bin -name test - search for all files in bin directory with name test

ps ax - show all running processes
ps aux - show all running processes with owner
top - show more detailed information about running processes
CTRL-z bg - put program into background
CTRL-z fg - put program into foreground

open -a <program name> - start a program from shell
kill -KILL <process id> - hard kill for process which not responding

As usually I'm not pretending for full list of BASH options - as it's really huge but those listed above are the most common for daily usage :)

Executing OSX terminal commands from Java

Couple days ago I've faced a situation when I needed to execute a pipeline of commands in OSX terminal from Java code and return result into program itself. You may say that there's a lot of information in the Internet (as a lot of us would say) but after spending some time it went out that those solutions are a bit incomplete for me :) So I'll add some information for those who might need that.

Problem:
- execute command in OSX terminal from Java program and return result from terminal back to the program;

Solution:

public static void main(String[] args) 
                             throws IOException, InterruptedException {
   try {
        Runtime runtime = Runtime.getRuntime();
        String[] cmd = {"/bin/bash", "-c", "Your command goes here"};

        Process process = runtime.exec(cmd);

        BufferedReader input = 
           new BufferedReader(new InputStreamReader(process.getInputStream()));

        String line;

        while ((line = input.readLine()) != null) {
                System.out.println(line);
        }

    } catch (Exception e) {
        System.out.println(e.toString());
        e.printStackTrace();
    }
}

And it's simply all. So You might want to create some kind of service class for that command creation and then simply pass it to runtime executor - it's up to You.

Before the end just one more important detail:

Problem:
- execute pipeline of commands (where we need to execute 2 or more) and only after that return a result back to the program

Solution:
- here I'll show an example of such pipeline:

"cd /Users/Projects/Test && git checkout master && git checkout -b master-test && git log"

So You can add as many of Your commands as You need just separate those with && and that will work.

That's probably all for that topic.