Tail a Log File on Windows & Linux

It turns out there are a bunch of people on StackOverflow looking for ways to tail a log file, but there don’t appear to be many resources for all the different tips and tools to do this. If you’re a Java or .NET developer, just getting started with tailing log files, or a seasoned developer who needs something quick and easy to set up and go, there are several options. In fact, there may be too many.

Check out some tools I found that make tailing a log file a walk in the park. Tailing multiple log files? Want to tail logs remotely from a web browser? This list covers a whole array of needs.

 

The Standard Linux Tail Command

The de facto standard for linux systems is the ever-handy “tail” command. Need I say more?

$ tail -f /var/log/syslog -f /var/log/myLog.log

• Quick and easy
• No additional installation required
• Can be used with multiple –f filenames in the same window as shown in the script example above
• Unix only. See Tail for Win32 at the bottom of this post for a port to Windows.

---

Less +F

Brian Storti (@brianstorti) offers an alternative method to the standard tail -f: less +F, which causes similar behavior as tail -f but allows for easily switching between navigation and watching mode. However, he notes that if you need to watch multiple files simultaneously, the standard tail -f offers better output.

• Easy to use
• Creates behavior comparable to tail -f
• Allows for easy switching between navigation and watching mode
• Not ideal for watching multiple files simultaneously

---

 

Windows Powershell

Powershell is one of the most overlooked windows apps for ops. This approach doesn’t have any extra features but can be perfect for opening a quick commandlet window and keeping an eye on the status of a file.

Use the following simple syntax to show the tail end of a log file in real-time.

Get-Content myTestLog.log –Wait

You can also filter the log right at the command line using regular expressions:

Get-Content myTestLog.log -wait | where { $_ -match “WARNING” }

• Quick and easy to get going
• Practically zero learning curve
• No additional installation necessary for newer windows machines
• Requires Windows Powershell (duh!)
• Slow for large files
• Basic functionality but some 3rd party extensions are available. For example, you need multiple cmdlet windows to monitor multiple files
• How-To Geek provides a step-by-step tutorial for getting tail-like functionality with Powershell that you may find useful.

---

 

Stackify’s Retrace

Retrace is an APM tool that provides all the support you need to monitor and optimize your applications, including enhanced log management which fully indexes and tags your logs.  Retrace also tails log files in real-time.

• Remotely tail log files via web browser
• Search all log files, including iis logs
• See how log files are trending and monitor specific logs
• Supported for Windows & Linux systems
• Free trial, low monthly cost

 

 

 

Vim (Using Tail Bundle Plugin)

Developed by Martin Krischik (@martin_krischik), this handy-dandy plugin for Vim allows you to use “the best tail plugin you can get.”

• Vim die-hards can tail log files without ever leaving their favorite editor!
• Multiple file tailing using tabs
• “Preview” window updated according to your Vim usage
• Read the open issues on the Google Code page before installing
• Check out Krischik’s other projects here

---

Emacs

To tail a file in Emacs (@emacs): start Emacs, hit M-x (Alt and x keys together), and type “tail-file”. Then, enter the filename to tail. The net result is that this will spawn an external tail -f process. Emacs is much more than a tool for tailing log files, however; it’s packed with other features and functionality ranging from project planning tools to debugging, a mail and news reader, calendar, and more.

• Customizable using Emacs Lisp code or a graphical interface
• Packaging system for downloading and installing extensions
• Unicode support for nearly all human scripts
• Built-in documentation
• Tutorial for new users
• Content-aware editing modes including syntax highlighting

---

 

MultiTail

Developed by Folkert van Heusden (@flok99), MultiTail was inspired by wtail, which was written by André Majorel. This is one of the more complete UNIX offerings, in my humble opinion. It’s a relatively newer tool compared to some others on this list, with a stable version released in February 2015.

• Uses wildcard matching to see if a more recently spawned logfile needs to be monitored
• Uses regular expressions for filtering
• Source code available in public Github repository
• All major UNIX platforms supported

---

 

BareTail

Developed by Bare Metal Software, BareTail is a free tool for monitoring log files in real-time. The “Bare” in the name might prompt some to ask, “How can you get any barer than regular Tail?” It turns out the name is a carryover from the software development group that built it, and this tool provides a color-coded GUI above and beyond good ‘ole Unix Tail.

• Developed for Windows
• Monitors multiple files using tabs
• Configurable highlighting
• Allows instant scrolling to any point in the file, even for large files
• Free version available; a registered license is $25. There’s also a BareTailPro, which is packed with even more features and offers a free demo.

---

LogTail

Developed by John Walker of Fourmilab (@Fourmilab), this tool doesn’t appear to have been supported in a long time (the website is dated 1997) and may not play well with the latest distros of Unix. You have been warned.

• Allows you to monitor multiple log files on multiple servers at once
• Automatically checks if the monitored process has spawned a fresh log file and adjusts monitoring accordingly
• Old script (circa 1997) may not play well with newer Unix distros/Perl patches (built withi Perl 4.0, patch level 36)
• UNIX only

---

 

TraceTool

Developed by Thierry Parent, TraceTool is a great option for .NET developers needing to build their own log tailing feature. The code comes with a lot of power and features, but it might not be as good a choice if you simply want to run a quick program and be off on your merry way. You can also check it out on SourceForge.

• Powerful tool with lots of customizability via code.
• There is a learning curve, depending on how far you want to take TraceTool. Check out the “TraceTool Overview” screenshot on the CodeProject page and you’ll see what I mean.
• Not a quick fix. If you simply want to open a quick executable and see the tail of a few log files, pick something else. This will take you some time to set up and get configured.
• With great power comes great responsibility. The CodeProject discussion section has comments from users experiencing several different kinds of problems. Yet despite any problems users have, they consistently ranked the page highly, with an average vote of 4.97 at the time of this writing.
• Source code is readily available for download, but you’ll need a .NET development environment setup to compile it.
• Windows only.

---

 

SnakeTail.net

SnakeTail.net is developed by SnakeNest. Looks can be deceiving – I thought this Google Project was long dead but was pleasantly surprised that bugs are still being fixed and features added as recently as September 2016.

• Low memory & CPU footprint even with large files
• Customizable shortcut keys to jump around files quickly
• Can tail a log directory where the latest log files are stored
• Windows only

---

 

Notepad++

Hardcore fans of Notepad++ (@Notepad_plus) often like to work in it all day, every day. Now you can tail a log file in Windows without ever leaving Notepad++ by using the Document Monitor plugin (granted – hardcore fans probably already know all about this!):

1. Open Notepad++ then from the top menu select “Plugins > Plugin Manager > Show Plugin Manager,” then check the option for “Document Monitor,” then click “Install.”
2. Notepad++ will prompt you to restart the program (not restart your computer).
3. Upon opening Notepad++ again, select “Plugins” and you should now see the “Document Monitor > Start to monitor” option. This will refresh the view of your document automatically every 3 seconds.

---

 

inotail

Developed by Tobias Klauser (@t_klauser), inotail is a basic tool with minimal options compared to the others on this list. But for those of you yearning for simplicity, this just might be the log tail tool you’re looking for. The most recent update, however, is from back in 2009, and the most recent version, inotail 0.5, was released in 2007.

• Git and GitHub repos available
• Uses the inotify API to determine if a file needs to be reread

---

 

Tail for Win32

Developed by Paul Perkins, Tail for Win32 is a Windows version of the UNIX ‘tail’ command, providing a quick and dirty way to use the Unix Tail command you’re used to on Windows systems. Many folks might consider this completely unnecessary on a windows system with the prevalence of Powershell these days, but it does provide a couple nice features you wouldn’t otherwise have:

• Highlighted keyword matching
• Can send email notifications via SMTP or MAPI when keywords are found
• The ability to watch multiple files in real-time simultaneously
• Can process files of any size on both local and networked drives
• Download Tail for Win32 on SourceForge

---

 

JLogTailer

JLogTailer is a Java log tailer with regular expression features that makes it possible to view any log file entries in real-time. It’s an easy-to-use tool that’s helpful when you need to see what’s going into the end of your log files as it happens while you tinker with your code. Additionally, you can use JLogTailer to tail multiple log files if needed.

• View log entries in real-time
• Simple to use
• Works for tailing multiple log files

---

WebTail

With so many programs generating log files, other tools are useful when you have direct access to the file system that stores each log. But what if you don’t have direct access to your log file? That’s where WebTail comes in, enabling you to tail any log file that can be viewed via a web server. In addition to allowing you to view entries the moment they’re appended to log files, WebTail requires less bandwidth than you’d otherwise use to download the entire file several times.

• Useful for tailing log files stored on web servers
• Save bandwidth by eliminating the need to download complete files multiple times
• Tail any log file viewable via a web server

---

MakeLogic Tail

An advanced tail -f command with GUI, MakeLogic Tail is the tail for Windows. It can be used to monitor the log files of various servers and comes with a variety of other intuitive and useful features.

• Shows the last several lines of growing log files
• Real-time monitoring
• Requires JRE 5.0
• Easy to use GUI
• Search current and all open log files
• Highlight select keywords in various colors
• Monitor most recently opened documents
• Cascade or Tile the log file windows

---

Wintail

Wintail is a free program created by Andy Hofle after he struggled with viewing log files in real-time with Windows Notepad. There was no program like tail -f in UNIX for Windows at the time, so he wrote his own. It’s a useful tool enabling you to have multiple tiled windows open simultaneously, and you can also pause updates to examine files more closely when needed.

• Supports large files over 2GB
• Error highlighting
• Drag-and-drop files you wish to monitor into the title or menu bar to open them
• Create shortcuts to frequently-monitored files
• Added support for Windows Server 2012 in November 2015

---

LogExpert

If you’re looking for an intuitive, easy-to-use tail application for Windows, LogExpert is a solid option offering search functionality, bookmarking, filter views, highlighting, time stamps, and other useful features that make monitoring a less frustrating task.

• Search functions including RegEx
• Flexible filter view
• Columnizers
• Highlight lines using search criteria
• Supports Unicode, SFTP, log4j XML files, and third-party plugins
• Plugin API for more log file data sources

---

glogg

A multi-platform GUI application for browsing and searching through long, complex log files, glogg can be described as a graphical, interactive combination of grep and less. An open-source tool released under the GPL by Nicolas Bonnefon, glogg functions on both Windows and Mac, making it a functional tool for any developer.

• Use regular expressions to search for events in log files
• Designed to help you spot and understand problems in the massive logs generated by embedded systems
• Also useful for sysadmins searching through logs on databases or servers
• Second window shows results from the current search
• Supports grep/egrep like regular expressions
• Colorizes logs and search results
• Follows logs written to a disk in real-time

Source: https://stackify.com/13-ways-to-tail-a-log-file-on-windows-unix/

Advertisements

Build Your Own OpenStreetMap Tile Server on Ubuntu 16.04

OpenStreetMap, aka, OSM is a user contributed world map that is freely editable. This tutorial will show you how to build your own OpenStreetMap tile server on Ubuntu 16.04. 2GB RAM, 40GB disk space should be the minimum OpenStreetMap server requirements.

Step 1: Upgrade Software

sudo apt update

sudo apt upgrade

Step 2: Install PostgreSQL Database Server with PostGIS

We will use PostgreSQL to store map data. PostGIS is a geospatial extenstion to PostgreSQL. Run the following commands to install them.

sudo apt install postgresql postgresql-contrib postgis postgresql-9.5-postgis-2.2

A user named

postgres will be created during the installation process. This allows the use of peer authentication. Let’s switch to the postgres user:

sudo -u postgres -i

Create a PostgreSQL database user osm.

createuser osm

Create a database named gis and at the same time make osm as the owner of the database. -E UTF8 specifies the character encoding scheme to be used in the database is UTF8.

createdb -E UTF8 -O osm gis

Create hstore and postgis extension.

psql -c "CREATE EXTENSION hstore;" -d gis

psql -c "CREATE EXTENSION postgis;" -d gis

Exit from the postgres user.

exit

Create osm user on your operating system so the tile server can run as osm user.

sudo adduser osm

Step 3: Download Map Stylesheet and Map Data

First switch to osm user

su - osm

Download the latest CartoCSS map stylesheets to the osm user’s home directory.

wget https://github.com/gravitystorm/openstreetmap-carto/archive/v2.41.0.tar.gz

Extract it.

tar xvf v2.41.0.tar.gz

Next, download map data to the osm user’s home directory. Use the below command to download the map data of the whole planet (32G).

wget -c http://planet.openstreetmap.org/pbf/planet-latest.osm.pbf

If you want a map of individual country or state, go to http://download.geofabrik.de. Also, BBBike.org provides extracts of more than 200 cities and regions world-wide in different formats.

For example, download the map data of Great Britain (847M).

wget -c http://download.geofabrik.de/europe/great-britain-latest.osm.pbf

Now exit from the osm user.

exit

Recommendations before Importing Map Data

Importing map data takes a lot of RAM. If your physical memory is small, you can easily add a swap file. First we use fallocate command to create a file. For example, create a file named swapfile with 2G capacity in root file system:

sudo fallocate -l 2G /swapfile

Then make sure only root can read and write to it.

sudo chmod 600 /swapfile

Format it to swap:

sudo mkswap /swapfile

Output:

Setting up swapspace version 1, size = 2097148 KiB
no label, UUID=h32b3e10-0779-4865-9ea0-6e2af8f3kea9

Enable the swap file

sudo swapon /swapfile

The import process can take some time. It’s recommended to configure SSH keepalive so that you don’t lose the SSH connection. It’s very easy to do. Just open the SSH client configuration file on your local Linux machine.

sudo nano /etc/ssh/ssh_config

And paste the following text at the end of the file.

ServerAliveInterval 60

Then save the file and connect to your Ubuntu 16.04 server

Step 4: Import the Map Data to PostgreSQL

To import map data, we need to install osm2pgsql which converts OpenStreetMap data to postGIS-enabled PostgreSQL databases.

sudo apt install osm2pgsql

Switch to osm user again.

su - osm

Run the following command to load map stylesheet and map data into the gis Database. Replace great-britain-latest.osm.pbf with your own map data file.

osm2pgsql --slim -d gis -C 3600 --hstore -S openstreetmap-carto-2.41.0/openstreetmap-carto.style great-britain-latest.osm.pbf

osm2gpsql will run in slim mode which is recommended over the normal mode. -d stands for --database. -C flag specify the cache size in MB. Bigger cache size results in faster import speed but you need to have enough RAM to use cache. -S flag specify the style file. And finally you need to specify the map data file.

Once the import is complete, exit from the osm user.

exit

Step 5: Install mod_tile

mod_tile is an Apache module that is required to serve tiles. Currently no binary package is available for Ubuntu. We can compile it from Github repository.

First install build dependency.

sudo apt install git autoconf libtool libmapnik-dev apache2-dev

Then clone the repository from Github.

git clone https://github.com/openstreetmap/mod_tile.git

cd mod_tile/

Compile and install

./autogen.sh
./configure
make
sudo make install
sudo make install-mod_tile

Step 6: Generate Mapnik Stylesheet

Install required packages.

sudo apt install curl unzip gdal-bin mapnik-utils node-carto

Switch to osm user.

su - osm

Cd into the carto style directory.

cd openstreetmap-carto-2.41.0/

Get shapefiles.

./get-shapefiles.sh

Now build the Mapnik xml stylesheet.

carto project.mml > style.xml

Exit from the osm user.

exit

Step 7: Configuring renderd

Edit renderd config file.

sudo nano /usr/local/etc/renderd.conf

In the [default] section, change the value of XML and HOST to the following.

XML=/home/osm/openstreetmap-carto-2.41.0/style.xml
HOST=localhost

In [mapnik] section, change the value of plugins_dir.

plugins_dir=/usr/lib/mapnik/3.0/input/

Save the file.

Install renderd init script by copying the sample init script.

sudo cp mod_tile/debian/renderd.init /etc/init.d/renderd

Grant execute permission.

sudo chmod a+x /etc/init.d/renderd

Edit the init script file

sudo nano /etc/init.d/renderd

Change the following variable.

DAEMON=/usr/local/bin/$NAME
DAEMON_ARGS="-c /usr/local/etc/renderd.conf"
RUNASUSER=osm

Save the file.

Create the following file and set osm the owner.

sudo mkdir -p /var/lib/mod_tile

sudo chown osm:osm /var/lib/mod_tile

Then start renderd service

sudo systemctl daemon-reload

sudo systemctl start renderd

sudo systemctl enable renderd

Step 8: Configure Apache

Install Apache web server

sudo apt install apache2

Create a module load file.

sudo nano /etc/apache2/mods-available/mod_tile.load

Paste the following line into the file.

LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so

Create a symlink.

sudo ln -s /etc/apache2/mods-available/mod_tile.load /etc/apache2/mods-enabled/

Then edit the default virtual host file.

sudo nano /etc/apache2/sites-enabled/000-default.conf

Past the following line in <VirtualHost *:80>

LoadTileConfigFile /usr/local/etc/renderd.conf
ModTileRenderdSocketName /var/run/renderd/renderd.sock
# Timeout before giving up for a tile to be rendered
ModTileRequestTimeout 0
# Timeout before giving up for a tile to be rendered that is otherwise missing
ModTileMissingRequestTimeout 30

Save and close the file. Restart Apache.

sudo systemctl restart apache2

Then in your web browser address bar, type

your-server-ip/osm_tiles/0/0/0.png

You should see the tile of world map. Congrats! You just successfully built your own OSM tile server.

Display Your Tiled Web Map

Tiled web map is also known as slippy map in OpenStreetMap terminology. There are two free and open source JavaScript map libraries you can use for your tile server: OpenLayer and Leaflet. The advantage of Leaflet is that it is simple to use and your map will be mobile-friendly.

OpenLayer

To display your slippy map with OpenLayer, first create a web folder.

sudo mkdir /var/www/osm

Then download JavaScript and CSS from openlayer.org and extract it to the web root folder.

Next, create the index.html file.

sudo nano /var/www/osm/index.html

Paste the following HTML code in the file. Replace red-colored text and adjust the longitude, latitude and zoom level according to your needs.

<!DOCTYPE html>
<html>
<head>
<title>Accessible Map</title>
<link rel="stylesheet" href="http://your-ip/ol.css" type="text/css">
http://your-ip/ol.js
<style>
  a.skiplink {
    position: absolute;
    clip: rect(1px, 1px, 1px, 1px);
    padding: 0;
    border: 0;
    height: 1px;
    width: 1px;
    overflow: hidden;
  }
  a.skiplink:focus {
    clip: auto;
    height: auto;
    width: auto;
    background-color: #fff;
    padding: 0.3em;
  }
  #map:focus {
    outline: #4A74A8 solid 0.15em;
  }
</style>
</head>
<body>
  <a class="skiplink" href="#map">Go to map</a>

<button id=”zoom-out”>Zoom out</button> <button id=”zoom-in”>Zoom in</button> var map = new ol.Map({ layers: [ new ol.layer.Tile({ source: new ol.source.OSM({ url: ‘http://your-ip/osm_tiles/{z}/{x}/{y}.png’ }) }) ], target: ‘map’, controls: ol.control.defaults({ attributionOptions: /** @type {olx.control.AttributionOptions} */ ({ collapsible: false }) }), view: new ol.View({ center: [244780.24508882355, 7386452.183179816], zoom:5 }) }); document.getElementById(‘zoom-out’).onclick = function() { var view = map.getView(); var zoom = view.getZoom(); view.setZoom(zoom – 1); }; document.getElementById(‘zoom-in’).onclick = function() { var view = map.getView(); var zoom = view.getZoom(); view.setZoom(zoom + 1); }; </body> </html>

Save and close the file. Now you can view your slippy map by typing your server IP address in browser.

your-ip/index.html           or          your-ip

Leaflet

To display your slippy map with Leftlet, first create a web folder.

sudo mkdir /var/www/osm

Then download JavaScript and CSS from leftletjs.com and extract it to the web root folder.

Next, create the index.html file.

sudo nano /var/www/osm/index.html

Paste the following HTML code in the file. Replace red-colored text and adjust the longitude, latitude and zoom level according to your needs.

<html>
<head>
<title>My first osm</title>
<link rel="stylesheet" type="text/css" href="leaflet.css"/>
http://leaflet.js
<style>
   #map{width:100%;height:100%}
</style>
</head>

<body>

var map = L.map(‘map’).setView([53.555,9.899],5); L.tileLayer(‘http://your-ip/osm_tiles/{z}/{x}/{y}.png’,{maxZoom:18}).addTo(map); </body> </html>

Save and close the file. Now you can view your slippy map by typing your server IP address in browser.

your-ip/index.html           or          your-ip

To pre-render tiles instead of rendering on the fly, use render_list command. Pre-rendered tiles will be cached in /var/lib/mod_tile directory. -z and -Z flag specify the zoom level.

render_list -m default -a -z 0 -Z 10

This tutorial is made available with the help from Miles B. Dyson.

There’s more than one way to kill a Unix process

Terminating processes on Unix systems is not quite an art, but there are sure a lot more options for how to select and terminate Unix processes than there are ways to skin a cat. In this post, we take a quick look at some of the commands and options that are likely to be the most useful.

Control sequences

The most obvious way to kill a process is probably to type Ctrl-C. This assumes, of course, that you’ve just started running it and that you’re still on the command line with the process running in the foreground. There are other control sequence options as well. You could suspend a process by using Ctrl-Z and then running a command such a kill %1 (depending on how many background processes you have running) to snuff it out.

• Ctrl-C sends SIGINT (interrupt)
• Ctrl-Z sends TSTP (terminal stop)
• Ctrl-\ sends SIGQUIT (terminate and dump core)
• Ctrl-T sends SIGINFO (show information), but this sequence is not supported on all Unix systems

Kill commands

Then, there are the various kill commands — including kill, pkill, xkill, and killall. The xkill command is a kill command for X Windows, so I won’t cover it here. The others are general purpose and provide very specific or fairly general process termination options.

kill

The kill command expects to be provided with a process ID and offers options such as -9 (often referred to as the “sure kill” option) to control what signal is sent to a running process. With the sure kill option, the kernel stops a process dead in its tracks giving it no time to execute a proper shutdown routine. Because this can result in data loss, this option should only be used when the process cannot otherwise be stopped.

$ kill -9 7911
$ kill -s 9 7911
$ kill -KILL 7911

Other commonly used kill options include:

-1 (-HUP) is the “hang up” option. This option is very safe. Most programs are built to listen for this signal and shut down in an orderly fashion if they do.

-2 (SIGINT) is the same signal that is used when you type Ctrl-C. It is generally safe, though some data loss is possible.

-15 (-TERM) is the “terminate” signal and the default. Like -1 (-HUP), it tells the process to shut down and is generally safe, though -1 (-HUP) is still the gentlest and most predictable.

You can get a complete list of the signals available to you using the kill -l(-l for “list”) command.

$ kill -l
 1) SIGHUP       2) SIGINT       3) SIGQUIT      4) SIGILL       5) SIGTRAP
 6) SIGABRT      7) SIGBUS       8) SIGFPE       9) SIGKILL     10) SIGUSR1
11) SIGSEGV     12) SIGUSR2     13) SIGPIPE     14) SIGALRM     15) SIGTERM
16) SIGSTKFLT   17) SIGCHLD     18) SIGCONT     19) SIGSTOP     20) SIGTSTP
21) SIGTTIN     22) SIGTTOU     23) SIGURG      24) SIGXCPU     25) SIGXFSZ
26) SIGVTALRM   27) SIGPROF     28) SIGWINCH    29) SIGIO       30) SIGPWR
31) SIGSYS      34) SIGRTMIN    35) SIGRTMIN+1  36) SIGRTMIN+2  37) SIGRTMIN+3
38) SIGRTMIN+4  39) SIGRTMIN+5  40) SIGRTMIN+6  41) SIGRTMIN+7  42) SIGRTMIN+8
43) SIGRTMIN+9  44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12 47) SIGRTMIN+13
48) SIGRTMIN+14 49) SIGRTMIN+15 50) SIGRTMAX-14 51) SIGRTMAX-13 52) SIGRTMAX-12
53) SIGRTMAX-11 54) SIGRTMAX-10 55) SIGRTMAX-9  56) SIGRTMAX-8  57) SIGRTMAX-7
58) SIGRTMAX-6  59) SIGRTMAX-5  60) SIGRTMAX-4  61) SIGRTMAX-3  62) SIGRTMAX-2
63) SIGRTMAX-1  64) SIGRTMAX

pkill

In its simplest form, the pkill command is handed the name of a process and sends it a SIGTERM (just like the kill command). The benefit is that you do a little less work, not having to look up the process ID, and the command does a little more. Of course, if you’re trying to kill a particular process and sixteen of them are running on the system, you’ll only be able to shoot down those that belong to you — unless you’re using your superuser powers. Sometimes you want to shut down one particular process. Sometimes you want to kill by name.

If we want to terminate a looping process called “loopy”, we can do that easily:

$ pkill loopy

We also could have done it this way, but that would even be more work.

$ ps -ef | grep loopy
shs       4310  4168  0 13:52 pts/0    00:00:00 /bin/bash ./loopy
shs       4314  4168  0 13:52 pts/0    00:00:00 grep --color=auto loopy
$ kill 4310

We also have the option of using the -u (username) option to kill by username, but note that this will attempt to terminate all processes being run by this user.

# pkill -u shs

Alternately, we can kill one specific process for one specific users.

# pkill -u shs loopy

Want to log off real quick and shut down everything you’re doing?

$ pkill -u `whoami`

This will kill all processes you might have running, even those running in the background.

The pgrep and pkill commands share a man page since they, after all, are just different sides of the same executable.

Options that work only for pgrep:

• -d delimiter between process IDs
• -l (include username in output)

options that work only for pkill

• -signal (specify signal to be sent)

The number of variety of signals that are available is quite mind-boggling. Under normal circumstances, however, you will probably only make use of these.

Signal Name    Signal Value  Behaviour
SIGHUP          1            Hangup
SIGKILL         9            Kill Signal
SIGTERM        15            Terminate

You can use the numeric or the more descriptive name in your kill commands — kill -9 and kill -KILL have the same effect.

But as you saw, if you read my post on pgrep, there are numerous other options available with pkill. You can specify the user, the effective userid, the terminal, the signal to be sent, the process group, just the oldest or newest instance of a process, etc.

sample commands

$ pkill -U intruder
$ pkill -U `whoami`
$ pkill -n myproc
$ pkill -f partial
# pkill -9 -P 1984
# pkill -g 123

You can also terminate a specific process by providing more information about the process you’re targeting. The -f option allows you to specify more than just the process name using a command pattern like this:

pkill -9 -f "COMMANDNAME -PARAMETERS"

killall

The killall command also provides the kill-by-name capability and, like kill and pkill, sends a SIGTERM by default. If some process seems to be replicating itself with wild abandon, you can shut it down with a command like killall repproc, killall -9 repproc, or killall -q -9 repproc. With the -q options, the command will run “quietly” and you won’t see output from the kill operation.

Options available with the killall command include:

-e exact match
-I ignore case
-g process group
-i interactive (ask for confirmation before killing)
-l list known signal names
-o older than
-q work quietly
-r interpret process name as regular expression
-s send the specified signal
-u specified user
-v verbose
-V show version
-w wait for all processes to die
-y younger than
-Z specify security context

Why signals?

Whenever you generate a signal using any form of kill command, the operating system interrupts the target process’ normal flow of execution. What happens next depends on the signal being sent.

The pkill and killall commands came about to make terminating process easier by identifying processes by name and other criteria. Going back to our “loopy” example, you could kill the process in one fell swoop, rather than looking up the process ID and then killing it. While the effect may be the same in both cases, the end result depends on whether other processes by the same name are running and whether you have sufficient access to terminate them.

What’s easiest depends on what you’re trying to do. Is your focus on one particular user, one particular process, or a wider range of users and processes? In general, using additional options allows your kill commands to be more targeted.

Source: http://www.computerworld.com/article/3174440/linux/theres-more-than-one-way-kill-a-unix-process.html

Nine ways to compare files on Unix

Sometimes you want to know if some files are different. Sometimes you want to how they’re different. Sometimes you might want to compare files that are compressed and sometimes you might want to compare executables. And, regardless of what you want to compare, you probably want to select the most convenient way to see those differences. The good news is that you have a lot more options than you probably imagine when you need to focus on file differences.

First: diff

The command most likely to come to mind for this task is diff. The diff command will show you the differences between two text files or tell you if two binaries are different, but it also has quite a few very useful options.

For text files, the diff command by default uses a format that shows the differences using < and > characters to represent the first and second of the two files and designations like 1c1 or 8d7 to report these differences in a format that could be used by the patch command.

$ diff file1 file2
1c1
< 0 top of file one
---
> 0 top of file 2
3c3
< 2
---
> 2 two tomatoes
6c6
< 5 five bananas
---
> 5
8d7
< 7 the end

If you just happen to have the patch command on your system, you can compare the two files, save the diff output to a file, and then use that file to force the second file to be the same as the first. The only time you’d likely want to do something like this is if you were trying to update files on a number of systems (rather than simply replace them) as the differences might be very small. You could do it like this:

Create the differences file:

$ diff file1 file2 > diffs

Use the differences file to make the seocnd file just like the first:

$ patch -i diffs file2

At this point, you’d have two identical files. Your file2 would be just like your file1. You could then use the diffs file on any number of systems to update the targeted file.

For most of us, this use of diff is probably not something we’d do very often.

If you only want to know if the files are different, you can try a simpler approach.

$ diff -q file1 file2
Files file1 and file2 differ

Second: side-by-side diff

If you want to see the differences between two files, but not the instructions that patch could use, you might like diff‘s side-by-side view. Note that lines with differences include a |.

$ diff -y file1 file2
0 top of file one                         | 0 top of file 2
1 one                                       1 one
2                                         | 2 two tomatoes
3 three                                     3 three
4                                           4
5 five bananas                            | 5
6                                           6
7 the end                                 <

Some of the most useful diff options are these:

-b	ignore white space differences
-B	ignore blank lines
-w	ignore all white space
-i	ignore case differences
-y	side-by-side

Third: top and bottom diff

The output from the diff command with the -c option displays the files sequentially with the different lines marked by exclamation points.

$ diff -c file1 file2
*** file1       2017-04-17 11:16:31.687059543 -0400
--- file2       2017-04-17 12:02:44.194623979 -0400
***************
*** 1,8 ****
! 0 top of file one
  1 one
! 2
  3 three
  4
! 5 five bananas
  6
- 7 the end
--- 1,7 ----
! 0 top of file 2
  1 one
! 2 two tomatoes
  3 three
  4
! 5
  6

Fourth: comparing binary files with diff

You can also use the diff command to compare binary files, but it will only tell you if the files are different unless you use the -s option.

$ diff /usr/bin/diff /usr/bin/cmp
Binary files /usr/bin/diff and /usr/bin/cmp differ
$ diff /usr/bin/diff mydiff
$ diff -s /usr/bin/diff mydiff
Files /usr/bin/diff and mydiff are identical

Fifth: cmp

The cmp command tells you if two files are different and where the first difference appears.

Here’s an example comparing text files:

$ cmp file1 file2
file1 file2 differ: byte 15, line 1

Here’s an example comparing binary files:

$ cmp /usr/bin/diff /usr/bin/cmp
/usr/bin/diff /usr/bin/cmp differ: byte 25, line 1

To illustrate just why we’re getting this particular response with the second command above, you can use the od command to view the top of each of these files. What you see below is the heading that is assigned to binary files. The content that represents the coding of these executables begins at the 25th byte.

$ od -bc /usr/bin/diff | head -4
0000000 177 105 114 106 002 001 001 000 000 000 000 000 000 000 000 000
        177   E   L   F 002 001 001  \0  \0  \0  \0  \0  \0  \0  \0  \0
0000020 002 000 076 000 001 000 000 000  024 076 100 000 000 000 000 000
        002  \0   >  \0 001  \0  \0  \0 024   >   @  \0  \0  \0  \0  \0
$ od -bc /usr/bin/cmp | head -4
0000000 177 105 114 106 002 001 001 000 000 000 000 000 000 000 000 000
        177   E   L   F 002 001 001  \0  \0  \0  \0  \0  \0  \0  \0  \0
0000020 002 000 076 000 001 000 000 000 060 047 100 000 000 000 000 000
        002  \0   >  \0 001  \0  \0  \0   0   '   @  \0  \0  \0  \0  \0

Sixth: comm

The comm command will display the differences in text files in a different format. In the example below, you can probably see that we’re looking at three separate columns of output. The first and second represent the first and second files. The third shows the lines which are the same in both of the two files.

$ comm file1 file2
        0 top of file 2
0 top of file one
                1 one
2
        2 two tomatoes
                3 three
                4
        5
5 five bananas
                6
7 the end

As you can see in the second example below, comparing a file to itself shows all of the output in column 3.

$ comm file1 file1
                0 top of file one
                1 one
                2
                3 three
                4
                5 five bananas
                6
                7 the end

Seventh: cksum

Checksums can also tell you if files are different. While this might not be advantageous when the files are on the same system, it can help a lot when they’re on different systems. By running the cksum command on each of the two systems, you can determine if they’re the same without having to move either of the files to the other system or share a file system or directory. Note that the cksum command is often used to verify the integrity of system files.

$ cksum /usr/bin/diff /usr/bin/cmp
3928148852 193512 /usr/bin/diff
4012608687 44096 /usr/bin/cmp

You can use an ssh command to get the checksum for the file on a remote system to see if they are the same or different.

$ cksum /usr/bin/diff
3928148852 193512 /usr/bin/diff
$ ssh remhost -l jdoe “/usr/bin/cksum /usr/bin/diff”
3928148852 193512 /usr/bin/diff

Eighth: comparing text files across systems

You can also compare files on two system without having to copy one of the file between systems or compare checksums by using a command like this one:

ssh remhost -l jdoe cat /home/jdoe/file2 | diff – file2

Ninth: diff3

The diff3 command works a lot like diff, but allows you to compare three files instead of only two. However, this command doesn’t have all the options that the diff has and, no, there’s no diff4, diff5, etc. Comparing files two at a time with your favorite comparison tool is probably a better strategy most of the time.

$ diff3 file1 file2 file3
====
1:1c
  0 top of file one
2:1c
  0 top of file 2
3:0a
====
1:3c
  2
2:3c
  2 two tomatoes
3:2c
  2 two
====
1:5,8c
  4
  5 five bananas
  6
  7 the end
2:5,7c
  4
  5
  6
3:4,7c
  4 four
  5 five
  6 six
  7 seven

Wrap-Up

There are a lot of choices at your disposal when you wanto to compare files on Unix systems. Hopefully you’ll find several that you really like using.

12 interesting Linux Commands that can make you laugh

1) Get a Random Quote with fortune command

One can get random quotes and funny predictions with the help of the fortune command. But you need to have fortune package  installed in your system.

To install Fortune for system based on apt or aptitude:

linuxtechi@linuxarena:~$ sudo apt-get install fortune

To install Fortune for yum based systems like CentOS / RHEL / Fedora, run the beneath command

[root@linuxtechi ~]# yum install fortune*

Once the installation is completed run fortune command. Example is shown below

2) Rev Command

The “rev” command reverses any string provided to it.whenever Linux is installed is installed automatically.

[root@linuxtechi ~]# rev
LinuxTechi
ihceTxuniL
Linux Rocks
skcoR xuniL
Opensource World
dlroW ecruosnepO
LinuxMint Ultimate Desktop
potkseD etamitlU tniMxuniL

One can even create a file with a list of names and just provide the file name next to the rev command and all the names in the file will be reversed.

[root@linuxtechi ~]# rev rev.txt
ihceTxuniL
skcoR syawla xuniL
dlrow ecruosnepO fo enobkcab eht si xuniL
ihceTxuniL moob moob
[root@linuxtechi ~]#

3) Yes Command

The Yes Command will keep on displaying yourtext for n number of times, until you intervene to stop it (CTRL + C)

synatx :

# yes <yourtext>

Example

[root@linuxtechi ~]# yes LinuxTechi.com
LinuxTechi.com
LinuxTechi.com
LinuxTechi.com
LinuxTechi.com
LinuxTechi.com
LinuxTechi.com
LinuxTechi.com
LinuxTechi.com
LinuxTechi.com^C
[root@linuxtechi ~]#

4) Turn Your Terminal into an Aquarium

Do you know that you can transform your linux system terminal into a beautiful sea aquarium with fishes and sea weeds? For that you need to install the term animator package, libcurses and  asciiaquarium package into your system.

To run the below steps successfully just make sure on your system ‘perl‘, ‘perl-Curses‘ and  ‘perl-devel‘ packages are installed

For yum based systems :

[root@linuxtechi ~]# yum install perl perl-devel perl-Curses

For Debian based systems

linuxtechi@linuxarena:~$ sudo apt-get install perl libcurses-perl

Follow the below steps to install and run this amazing Linux command:

First, make sure you’ve downloaded the term animator package

[root@linuxtechi ~]# wget http://search.cpan.org/CPAN/authors/id/K/KB/KBAUCOM/Term-Animation-2.6.tar.gz

Install the package using below steps

[root@linuxtechi ~]# tar zxpvf Term-Animation-2.6.tar.gz
[root@linuxtechi ~]# cd Term-Animation-2.6
[root@linuxtechi Term-Animation-2.6]# perl Makefile.PL && make && make test
[root@linuxtechi Term-Animation-2.6]# make install

Note: In Case you are facing any issues while executing above steps then  we can  use cpan console to install “Term Animation” Package as shown below

cpan[1]> install Term::Animation

Finally download and install the asciiaquarium package using below steps:

[root@linuxtechi ~]# wget http://www.robobunny.com/projects/asciiquarium/asciiquarium.tar.gz
[root@linuxtechi ~]# tar zxpvf asciiquarium.tar.gz
[root@linuxtechi ~]# cd asciiquarium_1.1/
[root@linuxtechi asciiquarium_1.1]# cp asciiquarium /usr/local/bin/
[root@linuxtechi asciiquarium_1.1]# chmod 755 /usr/local/bin/asciiquarium

Now run the command

[root@linuxtechi ~]# /usr/local/bin/asciiquarium

or

[root@linuxtechi ~]# asciiquarium

5) Toilet Command

As the name suggests, the functionality of this command is entirely different and don’t know why they named as toilet for this command. This command outputs large sized beautiful banners entirely built using small characters.

For Debian Based systems use below command to install toilet package

linuxtechi@linuxarena:~$ sudo apt-get install toilet

For stems like CentOS and RHEL use below commands to install toilet

[root@linuxtechi ~]# wget http://caca.zoy.org/raw-attachment/wiki/toilet/toilet-0.3.tar.gz
[root@linuxtechi ~]# tar zxpvf toilet-0.3.tar.gz
[root@linuxtechi ~]# cd toilet-0.3
[root@linuxtechi toilet-0.3]# ./configure && make && make install

Example:

linuxtechi@linuxarena:~$ toilet LinuxTechi Rocks

Toilet command offers wide range of options like fonts, colorful characters and filters. Use the man page of toilet command to explore all these options.

6) Set Your Terminal on Fire

Want to set your terminal on fire, don’t panic, it’s just some funny command that outputs fire like output using characters.

Install the package which provides aafire command on Debian like systems.

linuxtechi@linuxarena:~$ sudo apt-get install libaa-bin
linuxtechi@linuxarena:~$ aafire

7) Ask a Cow to Repeat What You Say

“cowsay” is another funny Linux command that displays your text input by a cow or any other animal of your choice

linuxtechi@linuxarena:~$ sudo apt-get install cowsay
linuxtechi@linuxarena:~$ cowsay Linux world Rocks

cowsay command can also be used with fortune command, example is shown below :

linuxtechi@linuxarena:~$ fortune | cowsay

8) Enter the World of Matrix

Relive the matrix experience with the help of cmatrix command.

cmatrix command is not available in the default OS installation, use the below command to install cmatrix package for Debian Based systems

linuxtechi@linuxarena:~$ sudo apt-get install cmatrix

Run the beneath command to start the matrix

linuxtechi@linuxarena:~$ cmatrix

9) Do You Love that Jerry Mouse

Are you getting bored with the same old black mouse pointer and want to have some fun with that. “Oneko” package comes to the rescue, which attaches “Jerry” the famed mouse to your mouse pointer and you can see that little cute mouse run along with your pointer. Pretty much Fun isn’t it ?

Install oneko package using below command on debian like systems

linuxtechi@linuxarena:~$ sudo apt-get install oneko

Once the installation is finished type the command “oneko”

10) Turn Your Terminal to Speak To You

Turn your speakers on as you can make your Linux terminal speak to you with the espeak command. Let’s first install epseak package

For Debian based systems

linuxtechi@linuxarena:~$ sudo apt-get install espeak

For yum based systems

[root@linuxtechi ~]# yum install espeak

Execute the espeak command followed by the text

linuxtechi@linuxarena:~$ espeak "LinuxTechi is the best Website for Linux guide and tutorials"

11)  Steam Locomotive in your Linux Terminal

You all know about the “ls” command used for displaying all the files in a folder, but do you know about “sl” ? It is a funny linux command that draws an ASCII steam locomotive in your Linux terminal screen.

‘sl’ command is not available by default, so let’s  first install it.

For Debian based systems

linuxtechi@linuxarena:~$ sudo apt-get install sl

for Yum based systems

[root@linuxtechi ~]# yum install sl

Run the ‘sl’ command and view the output

[root@linuxtechi ~]# sl

12) Watch Star Wars

Want to go back to your school days where you are dumbstruck with the characters of star wars, but you can’t see the movie here, but just see how Star wars text printed in ASCII characters.

[root@linuxtechi ~]# telnet towel.blinkenlights.nl

Wait for Couple of seconds , it will print Star wars text in ASCII format.

After couple of seconds , it will show below

To Stop or terminate starwars , simply press ctrl+] and after that telnet terminal will come and type quit there

That’s all from this article, I hope you guys have enjoyed these funny commands and if you feel any other command that should be included in this list, Please do let us know us.

Source: https://www.linuxtechi.com/12-interesting-linux-commands-make-you-laugh/

How To Install and Configure GitLab on Ubuntu 16.04

Introduction

GitLab CE, or Community Edition, is an open source application primarily used to host Git repositories, with additional development-related features like issue tracking. It is designed to be hosted using your own infrastructure, and provides flexibility in deploying as an internal repository store for your development team, publicly as a way to interface with users, or even open as a way for contributors to host their own projects.

The GitLab project makes it relatively straight forward to set up a GitLab instance on your own hardware with an easy installation mechanism. In this guide, we will cover how to install and configure GitLab on an Ubuntu 16.04 server.

Prerequisites

This tutorial will assume that you have access to a fresh Ubuntu 16.04 server. The published GitLab hardware requirements recommend using a server with:

• 2 cores
• 4GB of RAM

Although you may be able to get by with substituting some swap space for RAM, it is not recommended. For this guide we will assume that you have the above resources as a minimum.

In order to get started, you will a non-root user with sudo access configured on the server. It is also a good idea to set up a basic firewall to provide an additional layer of security. You can follow the steps in our Ubuntu 16.04 initial server setup guide to get this setup.

When you have satisfied the above prerequisites, continue on to start the installation procedure.

Installing the Dependencies

Before we can install GitLab itself, it is important to install some of the software that it leverages during installation and on an ongoing basis. Fortunately, all of the required software can be easily installed from Ubuntu’s default package repositories.

Since this is our first time using apt during this session, we can refresh the local package index and then install the dependencies by typing:

• sudo apt-get update
• sudo apt-get install ca-certificates curl openssh-server postfix

You will likely have some of this software installed already. For the postfix installation, select Internet Site when prompted. On the next screen, enter your server’s domain name or IP address to configure how the system will send mail.

Install GitLab

Now that the dependencies are in place, we can install GitLab itself. This is a straight forward process that leverages an installation script to configure your system with the GitLab repositories.

Move into the /tmp directory and then download the installation script:

• cd /tmp
• curl -LO https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh

Feel free to examine the downloaded script to ensure that you are comfortable with the actions that it will take. You can also find a hosted version of the script here:

• less /tmp/script.deb.sh

Once you are satisfied with the safety of the script, run the installer:

• sudo bash /tmp/script.deb.sh

The script will set up your server to use the GitLab maintained repositories. This lets you manage GitLab with the same package management tools you use for your other system packages. Once this is complete, you can install the actual GitLab application with apt:

• sudo apt-get install gitlab-ce

This will install the necessary components on your system. Before you can use the application, however, you need to run an initial configuration command:

• sudo gitlab-ctl reconfigure

This will initialize GitLab using information it can find about your server. This is a completely automated process, so you will not have to answer any prompts.

Adjusting the Firewall Rules

Before you can access the GitLab for the first time, you will need to ensure that your firewall rules are permissive enough to allow normal web traffic. If you followed the guide linked in the prerequisites, you will have a ufw firewall enabled.

View the current status of your active firewall by typing:

• sudo ufw status

Output

Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere                  
OpenSSH (v6)               ALLOW       Anywhere (v6)

As you can see, the current rules allow SSH traffic through, but access to other services is restricted. Since GitLab is a web application, we should allow HTTP access in.

Since the protocol to port mapping for HTTP is available in the /etc/services file, we can allow that traffic in by name. If you didn’t already have OpenSSH traffic enabled, you should allow that traffic now too:

• sudo ufw allow http
• sudo ufw allow OpenSSH

If you check the ufw status command again, you should see access configured to at least these two services:

• sudo ufw status

Output

Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere                  
80                         ALLOW       Anywhere                  
OpenSSH (v6)               ALLOW       Anywhere (v6)             
80 (v6)                    ALLOW       Anywhere (v6)

You should now be able to access the GitLab web interface.

Performing Initial Configuration Through the Web Interface

Now that GitLab is running and access is permitted, we can perform some initial configuration of the application through the web interface.

Logging In for the First Time

Visit the domain name of your GitLab server in your web browser:

http://gitlab_domain_or_IP

On your first time visiting, you should see an initial prompt to set a password for the administrative account:

In the initial password prompt, supply and confirm a secure password for the administrative account. Click on the Change your password button when you are finished.

You will be redirected to the conventional GitLab login page:

Here, you can log in with the password you just set. The credentials are:

• Username: root
• Password: [the password you set]

Enter these values into the fields for existing users and click the Sign in button. You will be signed into the application and taken to a landing page that prompts you to begin adding projects:

You can now make some simple changes to get GitLab set up the way you’d like.

Adjusting your Profile Settings

One of the first things that you should do after a fresh installation is to get your profile into better shape. GitLab selects some reasonable defaults, but these are not usually appropriate once you start using the software.

To make the necessary modifications, click on the user icon in the upper-right hand corner of the interface. In the drop down menu that appears, select Profile Settings:

You will be taken to the Profile section of your settings:

Adjust the Name and Email address from “Administrator” and “admin@example.com” to something more accurate. The name you select will be displayed to other users, while the email will be used for default avatar detection, notifications, Git actions through the interface, etc.

Click on the Update Profile settings button at the bottom when you are done:

A confirmation email will be sent to the address you provided. Follow the instructions in the email to confirm your account so that you can begin using it with GitLab.

Changing Your Account Name

Next, click on the Account menu item at the top of the page:

Here, you can find your private API token or configure two-factor authentication. However, the functionality we are interested in at the moment is the Change username section.

By default, the first administrative account is given the name root. Since this is a known account name, its more secure to change this a different name. You will still have administrative privileges; the only thing that will change is the name:

Click on the Update username button to make the change:

Next time you log in to the GitLab, remember to use your new username.

Add an SSH Key to your Account

In most cases, you will want to use SSH keys with Git to interact with your GitLab projects. To do this, you need to add your SSH public key to your GitLab account.

If you already have an SSH key pair created on your local computer, you can usually view the public key by typing:

• cat ~/.ssh/id_rsa.pub

You should see a large chunk of text, like this:

Output

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMuyMtMl6aWwqBCvQx7YXvZd7bCFVDsyln3yh5/8Pu23LW88VXfJgsBvhZZ9W0rPBGYyzE/TDzwwITvVQcKrwQrvQlYxTVbqZQDlmsC41HnwDfGFXg+QouZemQ2YgMeHfBzy+w26/gg480nC2PPNd0OG79+e7gFVrTL79JA/MyePBugvYqOAbl30h7M1a7EHP3IV5DQUQg4YUq49v4d3AvM0aia4EUowJs0P/j83nsZt8yiE2JEYR03kDgT/qziPK7LnVFqpFDSPC3MR3b8B354E9Af4C/JHgvglv2tsxOyvKupyZonbyr68CqSorO2rAwY/jWFEiArIaVuDiR9YM5 sammy@mydesktop

Copy this text and head back to the Profile Settings page in GitLab’s web interface.

If, instead, you get a message that looks like this, you do not yet have an SSH key pair configured on your machine:

Output

cat: /home/sammy/.ssh/id_rsa.pub: No such file or directory

If this is the case, you can create an SSH key pair by typing:

• ssh-keygen

Accept the defaults and optionally provide a password to secure the key locally:

Output

Generating public/private rsa key pair.
Enter file in which to save the key (/home/sammy/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/sammy/.ssh/id_rsa.
Your public key has been saved in /home/sammy/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:I8v5/M5xOicZRZq/XRcSBNxTQV2BZszjlWaIHi5chc0 sammy@gitlab.docsthat.work
The key's randomart image is:
+---[RSA 2048]----+
|          ..%o==B|
|           *.E =.|
|        . ++= B  |
|         ooo.o . |
|      . S .o  . .|
|     . + .. .   o|
|      +   .o.o ..|
|       o .++o .  |
|        oo=+     |
+----[SHA256]-----+

Once you have this, you can display your public key as above by typing:

• cat ~/.ssh/id_rsa.pub

Output

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMuyMtMl6aWwqBCvQx7YXvZd7bCFVDsyln3yh5/8Pu23LW88VXfJgsBvhZZ9W0rPBGYyzE/TDzwwITvVQcKrwQrvQlYxTVbqZQDlmsC41HnwDfGFXg+QouZemQ2YgMeHfBzy+w26/gg480nC2PPNd0OG79+e7gFVrTL79JA/MyePBugvYqOAbl30h7M1a7EHP3IV5DQUQg4YUq49v4d3AvM0aia4EUowJs0P/j83nsZt8yiE2JEYR03kDgT/qziPK7LnVFqpFDSPC3MR3b8B354E9Af4C/JHgvglv2tsxOyvKupyZonbyr68CqSorO2rAwY/jWFEiArIaVuDiR9YM5 sammy@mydesktop

Copy the block of text that’s displayed and head back to your Profile Settings in GitLab’s web interface.

Click on the SSH Keys menu item in the top menu bar:

In the provides space paste the public key you copied from your local machine. Give it a descriptive title, and click the Add key button:

You should now be able to manage your GitLab projects and repositories from your local machine without having to provide your GitLab account credentials.

Restrict or Disable Public Sign-ups (Optional)

You may have noticed that it is possible for anyone to sign up for an account when you visit your GitLab instance’s landing page. This may be what you want if you are looking to host public project. However, many times, more restrictive settings are desirable.

To begin, make your way to the administrative area by clicking on the wrench icon in the upper-right corner:

On the page that follows, you can see an overview of your GitLab instance as a whole. To adjust the settings, click on the gear icon in the upper-right corner and selecting Settings from the drop down menu that appears:

Note: At the time of writing (August, 2016), there is an outstanding issue with GitLab that affects the visibility of the settings icon at narrow screen widths. If you do not see the settings menu, try adjusting your browser window to full screen. You can also work around this issue by visiting your GitLab’s setting page directly:

http://gitlab_domain_or_IP/admin/application_settings

You will be taken to the global settings for your GitLab instance. Here, you can adjust a number of settings that affect whether new users can sign up and what their level of access will be.

Disabling Sign-ups

If you wish to disable sign-ups completely (you can still manually create accounts for new users), visit the Sign-up Restrictions section.

Deselect the Sign-up enabled check box:

Scroll down to the bottom and click on the Save button:

The sign-up section should now be removed from the GitLab landing page.

Restrict Sign-ups By Domain

If you are using GitLab as part of an organization that provides email addresses associated with a domain, you can restrict sign-ups by domain instead of completely disabling them.

In the Sign-up Restrictions section, first select the Send confirmation email on sign-up box only allow users to log in after they’ve confirmed their email.

Next, add your domain or domains to the Whitelisted domains for sign-ups box, one per line. You can use the asterisk “*” to specify wildcard domains:

Scroll down to the bottom and click on the Save button:

The sign-up section should now be removed from the GitLab landing page.

Restricting Project Creation

By default, new users can create up to 10 projects. If you wish to allow new users from the outside for visibility and participation, but want to restrict their access to creating new projects, you can do so in the Account and Limit Settings section.

Inside, you can change the Default projects limit to 0 to completely disable new users from creating projects:

New users can still be added to projects manually and will have access to internal or public projects created by other users.

Scroll down to the bottom and click on the Save button:

New users will now be able to create accounts, but unable to create projects.

Conclusion

You should now have a working GitLab instance hosted on your own server. You can begin to import or create new projects and configure the appropriate level of access for your team.

Before going much further, you should configure SSL encryption for your GitLab server. Without SSL protection, your passwords and private details can be intercepted by anyone on the network. Luckily, with projects like Let’s Encrypt, it is relatively straight forward to get a free, SSL certificate trusted by browsers. To learn how to get your GitLab instance set up with an SSL certificate, follow our guide on how to secure GitLab with Let’s Encrypt.

Source: https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-gitlab-on-ubuntu-16-04

A cheat sheet list of 10 nmap commands that are extremely useful for every Systems or Network Administrator

Recently I was compiling a list of Linux commands that every sysadmin should know. One of the first commands that came to mind was nmap.

nmap is a powerful network scanner used to identify systems and services. nmap was originally developed with network security in mind, it is a tool that was designed to find vulnerabilities within a network. nmap is more than just a simple port scanner though, you can use nmap to find specific versions of services, certain OS types, or even find that pesky printer someone put on your network without telling you.

nmap can be used for good and for evil, today we will cover some common situations where nmap makes life easier for sysadmins which is generally good. Even if some Sysadmins are evil…

Discover IP’s in a subnet (no root)

 $ nmap -sP 192.168.0.0/24
 Starting Nmap 5.21 ( http://nmap.org ) at 2013-02-24 09:37 MST
 Nmap scan report for 192.168.0.1
 Host is up (0.0010s latency).
 Nmap scan report for 192.168.0.95
 Host is up (0.0031s latency).
 Nmap scan report for 192.168.0.110
 Host is up (0.0018s latency).

This is one of the simplest uses of nmap. This command is commonly refereed to as a “ping scan”, and tells nmap to send an icmp echo request, TCP SYN to port 443, TCP ACK to port 80 and icmp timestamp request to all hosts in the specified subnet. nmap will simply return a list of ip’s that responded. Unlike many nmap commands this particular one does not require root privileges, however when executed by root nmap will also by default send arp requests to the subnet.

Scan for open ports (no root)

 $ nmap 192.168.0.0/24
 Starting Nmap 5.21 ( http://nmap.org ) at 2013-02-24 09:23 MST
 Nmap scan report for 192.168.0.1
 Host is up (0.0043s latency).
 Not shown: 998 closed ports
 PORT STATE SERVICE
 80/tcp open http
 443/tcp open https

This scan is the default scan for nmap and can take some time to generate. With this scan nmap will attempt a TCP SYN connection to 1000 of the most common ports as well as an icmp echo request to determine if a host is up. nmap will also perform a DNS reverse lookup on the identified ip’s as this can sometimes be useful information.

Identify the Operating System of a host (requires root)

 # nmap -O 192.168.0.164
 Starting Nmap 5.21 ( http://nmap.org ) at 2013-02-24 09:49 MST
 Nmap scan report for 192.168.0.164
 Host is up (0.00032s latency).
 Not shown: 996 closed ports
 PORT STATE SERVICE
 88/tcp open kerberos-sec
 139/tcp open netbios-ssn
 445/tcp open microsoft-ds
 631/tcp open ipp
 MAC Address: 00:00:00:00:00:00 (Unknown)
 Device type: general purpose
 Running: Apple Mac OS X 10.5.X
 OS details: Apple Mac OS X 10.5 - 10.6 (Leopard - Snow Leopard) (Darwin 9.0.0b5 - 10.0.0)
 Network Distance: 1 hop

With the -O option nmap will try to guess the targets operating system. This is accomplished by utilizing information that nmap is already getting through the TCP SYN port scan. This is usually a best guess but can actually be fairly accurate. The operating system scan however does require root privileges.

Identify Hostnames (no root)

 $ nmap -sL 192.168.0.0/24
 Starting Nmap 5.21 ( http://nmap.org ) at 2013-02-24 09:59 MST
 Nmap scan report for 192.168.0.0
 Nmap scan report for router.local (192.168.0.1)
 Nmap scan report for fakehost.local (192.168.0.2)
 Nmap scan report for another.fakehost.local (192.168.0.3)

This is one of the most subtle commands of nmap, the -sL flag tells nmap to do a simple DNS query for the specified ip. This allows you to find hostnames for all of the ip’s in a subnet without having send a packet to the individual hosts themselves.

Hostname information can tell you a lot more about a network than you would think, for instance if you labeled your Active Directory Servers with ads01.domain.com you shouldn’t be surprised if someone guesses its use.

TCP Syn and UDP Scan (requires root)

 # nmap -sS -sU -PN 192.168.0.164
 Starting Nmap 5.21 ( http://nmap.org ) at 2013-02-24 13:25 MST
 Nmap scan report for 192.168.0.164
 Host is up (0.00029s latency).
 Not shown: 1494 closed ports, 496 filtered ports
 PORT STATE SERVICE
 88/tcp open kerberos-sec
 139/tcp open netbios-ssn
 445/tcp open microsoft-ds
 631/tcp open ipp
 88/udp open|filtered kerberos-sec
 123/udp open ntp
 137/udp open netbios-ns
 138/udp open|filtered netbios-dgm
 631/udp open|filtered ipp
 5353/udp open zeroconf

The TCP SYN and UDP scan will take a while to generate but is fairly unobtrusive and stealthy. This command will check about 2000 common tcp and udp ports to see if they are responding. When you use the -Pn flag this tells nmap to skip the ping scan and assume the host is up. This can be useful when there is a firewall that might be preventing icmp replies.

TCP SYN and UDP scan for all ports (requires root)

 # nmap -sS -sU -PN -p 1-65535 192.168.0.164
 Starting Nmap 5.21 ( http://nmap.org ) at 2013-02-24 10:18 MST
 Nmap scan report for 192.168.0.164
 Host is up (0.00029s latency).
 Not shown: 131052 closed ports
 PORT STATE SERVICE
 88/tcp open kerberos-sec
 139/tcp open netbios-ssn
 445/tcp open microsoft-ds
 631/tcp open ipp
 17500/tcp open unknown
 88/udp open|filtered kerberos-sec
 123/udp open ntp
 137/udp open netbios-ns
 138/udp open|filtered netbios-dgm
 631/udp open|filtered ipp
 5353/udp open zeroconf
 17500/udp open|filtered unknown
 51657/udp open|filtered unknown
 54658/udp open|filtered unknown
 56128/udp open|filtered unknown
 57798/udp open|filtered unknown
 58488/udp open|filtered unknown
 60027/udp open|filtered unknown

This command is the same as above however by specifying the full port range from 1 to 65535 nmap will scan to see if the host is listening on all available ports. You can use the port range specification on any scan that performs a port scan.

TCP Connect Scan (no root)

 $ nmap -sT 192.168.0.164
 Starting Nmap 5.21 ( http://nmap.org ) at 2013-02-24 12:48 MST
 Nmap scan report for 192.168.0.164
 Host is up (0.0014s latency).
 Not shown: 964 closed ports, 32 filtered ports
 PORT STATE SERVICE
 88/tcp open kerberos-sec
 139/tcp open netbios-ssn
 445/tcp open microsoft-ds
 631/tcp open ipp

This command is similar to the TCP SYN scan however rather than sending a SYN packet and reviewing the headers it will ask the OS to establish a TCP connection to the 1000 common ports.

Aggressively Scan Hosts (no root)

 $ nmap -T4 -A 192.168.0.0/24
 Nmap scan report for 192.168.0.95
 Host is up (0.00060s latency).
 Not shown: 996 closed ports
 PORT STATE SERVICE VERSION
 22/tcp open ssh OpenSSH 5.9p1 Debian 5ubuntu1 (protocol 2.0)
 | ssh-hostkey: 1024 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:6c (DSA)
 |_2048 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:6c (RSA)
 80/tcp open http nginx 1.1.19
 |_http-title: 403 Forbidden
 |_http-methods: No Allow or Public header in OPTIONS response (status code 405)
 111/tcp open rpcbind
 | rpcinfo:
 | program version port/proto service
 | 100000 2,3,4 111/tcp rpcbind
 | 100000 2,3,4 111/udp rpcbind
 | 100003 2,3,4 2049/tcp nfs
 | 100003 2,3,4 2049/udp nfs
 | 100005 1,2,3 46448/tcp mountd
 | 100005 1,2,3 52408/udp mountd
 | 100021 1,3,4 35394/udp nlockmgr
 | 100021 1,3,4 57150/tcp nlockmgr
 | 100024 1 49363/tcp status
 | 100024 1 51515/udp status
 | 100227 2,3 2049/tcp nfs_acl
 |_ 100227 2,3 2049/udp nfs_acl
 2049/tcp open nfs (nfs V2-4) 2-4 (rpc #100003)
 Service Info: OS: Linux; CPE: cpe:/o:linux:kernel

Unlike some of the earlier commands this command is very aggressive and very obtrusive. The -A simply tells nmap to perform OS checking and version checking. The -T4 is for the speed template, these templates are what tells nmap how quickly to perform the scan. The speed template ranges from 0 for slow and stealthy to 5 for fast and obvious.

Fast Scan (no root)

 $ nmap -T4 -F 192.168.0.164
 Starting Nmap 6.01 ( http://nmap.org ) at 2013-02-24 12:49 MST
 Nmap scan report for 192.168.0.164
 Host is up (0.00047s latency).
 Not shown: 96 closed ports
 PORT STATE SERVICE
 88/tcp open kerberos-sec
 139/tcp open netbios-ssn
 445/tcp open microsoft-ds
 631/tcp open ipp

This scan limits the scan to the most common 100 ports, if you simply want to know some potential hosts with ports open that shouldn’t be this is a quick and dirty command to use.

Verbose

 $ nmap -T4 -A -v 192.168.0.164
 Starting Nmap 6.01 ( http://nmap.org ) at 2013-02-24 12:50 MST
 NSE: Loaded 93 scripts for scanning.
 NSE: Script Pre-scanning.
 Initiating Ping Scan at 12:50
 Scanning 192.168.0.164 [2 ports]
 Completed Ping Scan at 12:50, 0.00s elapsed (1 total hosts)
 Initiating Parallel DNS resolution of 1 host. at 12:50
 Completed Parallel DNS resolution of 1 host. at 12:50, 0.01s elapsed
 Initiating Connect Scan at 12:50
 Scanning 192.168.0.164 [1000 ports]
 Discovered open port 139/tcp on 192.168.0.164
 Discovered open port 445/tcp on 192.168.0.164
 Discovered open port 88/tcp on 192.168.0.164
 Discovered open port 631/tcp on 192.168.0.164
 Completed Connect Scan at 12:50, 5.22s elapsed (1000 total ports)
 Initiating Service scan at 12:50
 Scanning 4 services on 192.168.0.164
 Completed Service scan at 12:51, 11.00s elapsed (4 services on 1 host)
 NSE: Script scanning 192.168.0.164.
 Initiating NSE at 12:51
 Completed NSE at 12:51, 12.11s elapsed
 Nmap scan report for 192.168.0.164
 Host is up (0.00026s latency).
 Not shown: 996 closed ports
 PORT STATE SERVICE VERSION
 88/tcp open kerberos-sec Mac OS X kerberos-sec
 139/tcp open netbios-ssn Samba smbd 3.X (workgroup: WORKGROUP)
 445/tcp open netbios-ssn Samba smbd 3.X (workgroup: WORKGROUP)
 631/tcp open ipp CUPS 1.4
 | http-methods: GET HEAD OPTIONS POST PUT
 | Potentially risky methods: PUT
 |_See http://nmap.org/nsedoc/scripts/http-methods.html
 | http-robots.txt: 1 disallowed entry
 |_/
 Service Info: OS: Mac OS X; CPE: cpe:/o:apple:mac_os_x

By adding verbose to a majority of the commands above you get a better insight into what nmap is doing; for some scans verbosity will provide additional details that the report does not provide.

While these are 10 very useful nmap commands I am sure there are some more handy nmap examples out there. If you have one to add to this list feel free to drop it into a comment.

Source: http://bencane.com/2013/02/25/10-nmap-commands-every-sysadmin-should-know/