How To Install and Configure GitLab on Ubuntu 16.04

Introduction

GitLab CE, or Community Edition, is an open source application primarily used to host Git repositories, with additional development-related features like issue tracking. It is designed to be hosted using your own infrastructure, and provides flexibility in deploying as an internal repository store for your development team, publicly as a way to interface with users, or even open as a way for contributors to host their own projects.

The GitLab project makes it relatively straight forward to set up a GitLab instance on your own hardware with an easy installation mechanism. In this guide, we will cover how to install and configure GitLab on an Ubuntu 16.04 server.

Prerequisites

This tutorial will assume that you have access to a fresh Ubuntu 16.04 server. The published GitLab hardware requirements recommend using a server with:

  • 2 cores
  • 4GB of RAM

Although you may be able to get by with substituting some swap space for RAM, it is not recommended. For this guide we will assume that you have the above resources as a minimum.

In order to get started, you will a non-root user with sudo access configured on the server. It is also a good idea to set up a basic firewall to provide an additional layer of security. You can follow the steps in our Ubuntu 16.04 initial server setup guide to get this setup.

When you have satisfied the above prerequisites, continue on to start the installation procedure.

Installing the Dependencies

Before we can install GitLab itself, it is important to install some of the software that it leverages during installation and on an ongoing basis. Fortunately, all of the required software can be easily installed from Ubuntu’s default package repositories.

Since this is our first time using apt during this session, we can refresh the local package index and then install the dependencies by typing:

  • sudo apt-get update
  • sudo apt-get install ca-certificates curl openssh-server postfix

You will likely have some of this software installed already. For the postfix installation, select Internet Site when prompted. On the next screen, enter your server’s domain name or IP address to configure how the system will send mail.

Install GitLab

Now that the dependencies are in place, we can install GitLab itself. This is a straight forward process that leverages an installation script to configure your system with the GitLab repositories.

Move into the /tmp directory and then download the installation script:

Feel free to examine the downloaded script to ensure that you are comfortable with the actions that it will take. You can also find a hosted version of the script here:

  • less /tmp/script.deb.sh

Once you are satisfied with the safety of the script, run the installer:

  • sudo bash /tmp/script.deb.sh

The script will set up your server to use the GitLab maintained repositories. This lets you manage GitLab with the same package management tools you use for your other system packages. Once this is complete, you can install the actual GitLab application with apt:

  • sudo apt-get install gitlab-ce

This will install the necessary components on your system. Before you can use the application, however, you need to run an initial configuration command:

  • sudo gitlab-ctl reconfigure

This will initialize GitLab using information it can find about your server. This is a completely automated process, so you will not have to answer any prompts.

Adjusting the Firewall Rules

Before you can access the GitLab for the first time, you will need to ensure that your firewall rules are permissive enough to allow normal web traffic. If you followed the guide linked in the prerequisites, you will have a ufw firewall enabled.

View the current status of your active firewall by typing:

  • sudo ufw status
Output
Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere                  
OpenSSH (v6)               ALLOW       Anywhere (v6)

As you can see, the current rules allow SSH traffic through, but access to other services is restricted. Since GitLab is a web application, we should allow HTTP access in.

Since the protocol to port mapping for HTTP is available in the /etc/services file, we can allow that traffic in by name. If you didn’t already have OpenSSH traffic enabled, you should allow that traffic now too:

  • sudo ufw allow http
  • sudo ufw allow OpenSSH

If you check the ufw status command again, you should see access configured to at least these two services:

  • sudo ufw status
Output
Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere                  
80                         ALLOW       Anywhere                  
OpenSSH (v6)               ALLOW       Anywhere (v6)             
80 (v6)                    ALLOW       Anywhere (v6)

You should now be able to access the GitLab web interface.

Performing Initial Configuration Through the Web Interface

Now that GitLab is running and access is permitted, we can perform some initial configuration of the application through the web interface.

Logging In for the First Time

Visit the domain name of your GitLab server in your web browser:

http://gitlab_domain_or_IP

On your first time visiting, you should see an initial prompt to set a password for the administrative account:

GitLab initial password set prompt

In the initial password prompt, supply and confirm a secure password for the administrative account. Click on the Change your password button when you are finished.

You will be redirected to the conventional GitLab login page:

GitLab first sign in prompt

Here, you can log in with the password you just set. The credentials are:

  • Username: root
  • Password: [the password you set]

Enter these values into the fields for existing users and click the Sign in button. You will be signed into the application and taken to a landing page that prompts you to begin adding projects:

GitLab initial login landing page

You can now make some simple changes to get GitLab set up the way you’d like.

Adjusting your Profile Settings

One of the first things that you should do after a fresh installation is to get your profile into better shape. GitLab selects some reasonable defaults, but these are not usually appropriate once you start using the software.

To make the necessary modifications, click on the user icon in the upper-right hand corner of the interface. In the drop down menu that appears, select Profile Settings:

GitLab profile settings button

You will be taken to the Profile section of your settings:

GitLab profile settings page

Adjust the Name and Email address from “Administrator” and “admin@example.com” to something more accurate. The name you select will be displayed to other users, while the email will be used for default avatar detection, notifications, Git actions through the interface, etc.

Click on the Update Profile settings button at the bottom when you are done:

GitLab update profile settings button

A confirmation email will be sent to the address you provided. Follow the instructions in the email to confirm your account so that you can begin using it with GitLab.

Changing Your Account Name

Next, click on the Account menu item at the top of the page:

GitLab account menu item

Here, you can find your private API token or configure two-factor authentication. However, the functionality we are interested in at the moment is the Change username section.

By default, the first administrative account is given the name root. Since this is a known account name, its more secure to change this a different name. You will still have administrative privileges; the only thing that will change is the name:

GitLab change username section

Click on the Update username button to make the change:

GitLab update username button

Next time you log in to the GitLab, remember to use your new username.

Add an SSH Key to your Account

In most cases, you will want to use SSH keys with Git to interact with your GitLab projects. To do this, you need to add your SSH public key to your GitLab account.

If you already have an SSH key pair created on your local computer, you can usually view the public key by typing:

  • cat ~/.ssh/id_rsa.pub

You should see a large chunk of text, like this:

Output
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMuyMtMl6aWwqBCvQx7YXvZd7bCFVDsyln3yh5/8Pu23LW88VXfJgsBvhZZ9W0rPBGYyzE/TDzwwITvVQcKrwQrvQlYxTVbqZQDlmsC41HnwDfGFXg+QouZemQ2YgMeHfBzy+w26/gg480nC2PPNd0OG79+e7gFVrTL79JA/MyePBugvYqOAbl30h7M1a7EHP3IV5DQUQg4YUq49v4d3AvM0aia4EUowJs0P/j83nsZt8yiE2JEYR03kDgT/qziPK7LnVFqpFDSPC3MR3b8B354E9Af4C/JHgvglv2tsxOyvKupyZonbyr68CqSorO2rAwY/jWFEiArIaVuDiR9YM5 sammy@mydesktop

Copy this text and head back to the Profile Settings page in GitLab’s web interface.

If, instead, you get a message that looks like this, you do not yet have an SSH key pair configured on your machine:

Output
cat: /home/sammy/.ssh/id_rsa.pub: No such file or directory

If this is the case, you can create an SSH key pair by typing:

  • ssh-keygen

Accept the defaults and optionally provide a password to secure the key locally:

Output
Generating public/private rsa key pair.
Enter file in which to save the key (/home/sammy/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/sammy/.ssh/id_rsa.
Your public key has been saved in /home/sammy/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:I8v5/M5xOicZRZq/XRcSBNxTQV2BZszjlWaIHi5chc0 sammy@gitlab.docsthat.work
The key's randomart image is:
+---[RSA 2048]----+
|          ..%o==B|
|           *.E =.|
|        . ++= B  |
|         ooo.o . |
|      . S .o  . .|
|     . + .. .   o|
|      +   .o.o ..|
|       o .++o .  |
|        oo=+     |
+----[SHA256]-----+

Once you have this, you can display your public key as above by typing:

  • cat ~/.ssh/id_rsa.pub
Output
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMuyMtMl6aWwqBCvQx7YXvZd7bCFVDsyln3yh5/8Pu23LW88VXfJgsBvhZZ9W0rPBGYyzE/TDzwwITvVQcKrwQrvQlYxTVbqZQDlmsC41HnwDfGFXg+QouZemQ2YgMeHfBzy+w26/gg480nC2PPNd0OG79+e7gFVrTL79JA/MyePBugvYqOAbl30h7M1a7EHP3IV5DQUQg4YUq49v4d3AvM0aia4EUowJs0P/j83nsZt8yiE2JEYR03kDgT/qziPK7LnVFqpFDSPC3MR3b8B354E9Af4C/JHgvglv2tsxOyvKupyZonbyr68CqSorO2rAwY/jWFEiArIaVuDiR9YM5 sammy@mydesktop

Copy the block of text that’s displayed and head back to your Profile Settings in GitLab’s web interface.

Click on the SSH Keys menu item in the top menu bar:

GitLab SSH Keys menu item

In the provides space paste the public key you copied from your local machine. Give it a descriptive title, and click the Add key button:

GitLab add SSH Key

You should now be able to manage your GitLab projects and repositories from your local machine without having to provide your GitLab account credentials.

Restrict or Disable Public Sign-ups (Optional)

You may have noticed that it is possible for anyone to sign up for an account when you visit your GitLab instance’s landing page. This may be what you want if you are looking to host public project. However, many times, more restrictive settings are desirable.

To begin, make your way to the administrative area by clicking on the wrench icon in the upper-right corner:

GitLab administrative area button

On the page that follows, you can see an overview of your GitLab instance as a whole. To adjust the settings, click on the gear icon in the upper-right corner and selecting Settings from the drop down menu that appears:

Note: At the time of writing (August, 2016), there is an outstanding issue with GitLab that affects the visibility of the settings icon at narrow screen widths. If you do not see the settings menu, try adjusting your browser window to full screen. You can also work around this issue by visiting your GitLab’s setting page directly:

http://gitlab_domain_or_IP/admin/application_settings

GitLab administrative settings button

You will be taken to the global settings for your GitLab instance. Here, you can adjust a number of settings that affect whether new users can sign up and what their level of access will be.

Disabling Sign-ups

If you wish to disable sign-ups completely (you can still manually create accounts for new users), visit the Sign-up Restrictions section.

Deselect the Sign-up enabled check box:

GitLab deselect sign-ups enabled

Scroll down to the bottom and click on the Save button:

GitLab save settings button

The sign-up section should now be removed from the GitLab landing page.

Restrict Sign-ups By Domain

If you are using GitLab as part of an organization that provides email addresses associated with a domain, you can restrict sign-ups by domain instead of completely disabling them.

In the Sign-up Restrictions section, first select the Send confirmation email on sign-up box only allow users to log in after they’ve confirmed their email.

Next, add your domain or domains to the Whitelisted domains for sign-ups box, one per line. You can use the asterisk “*” to specify wildcard domains:

GitLab restrict sign-ups by domain

Scroll down to the bottom and click on the Save button:

GitLab save settings button

The sign-up section should now be removed from the GitLab landing page.

Restricting Project Creation

By default, new users can create up to 10 projects. If you wish to allow new users from the outside for visibility and participation, but want to restrict their access to creating new projects, you can do so in the Account and Limit Settings section.

Inside, you can change the Default projects limit to 0 to completely disable new users from creating projects:

GitLab set projects to zero

New users can still be added to projects manually and will have access to internal or public projects created by other users.

Scroll down to the bottom and click on the Save button:

GitLab save settings button

New users will now be able to create accounts, but unable to create projects.

Conclusion

You should now have a working GitLab instance hosted on your own server. You can begin to import or create new projects and configure the appropriate level of access for your team.

Before going much further, you should configure SSL encryption for your GitLab server. Without SSL protection, your passwords and private details can be intercepted by anyone on the network. Luckily, with projects like Let’s Encrypt, it is relatively straight forward to get a free, SSL certificate trusted by browsers. To learn how to get your GitLab instance set up with an SSL certificate, follow our guide on how to secure GitLab with Let’s Encrypt.

Source: https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-gitlab-on-ubuntu-16-04

Advertisements

A cheat sheet list of 10 nmap commands that are extremely useful for every Systems or Network Administrator

Recently I was compiling a list of Linux commands that every sysadmin should know. One of the first commands that came to mind was nmap.

nmap is a powerful network scanner used to identify systems and services. nmap was originally developed with network security in mind, it is a tool that was designed to find vulnerabilities within a network. nmap is more than just a simple port scanner though, you can use nmap to find specific versions of services, certain OS types, or even find that pesky printer someone put on your network without telling you.

nmap can be used for good and for evil, today we will cover some common situations where nmap makes life easier for sysadmins which is generally good. Even if some Sysadmins are evil…

Discover IP’s in a subnet (no root)

 $ nmap -sP 192.168.0.0/24
 Starting Nmap 5.21 ( http://nmap.org ) at 2013-02-24 09:37 MST
 Nmap scan report for 192.168.0.1
 Host is up (0.0010s latency).
 Nmap scan report for 192.168.0.95
 Host is up (0.0031s latency).
 Nmap scan report for 192.168.0.110
 Host is up (0.0018s latency).

This is one of the simplest uses of nmap. This command is commonly refereed to as a “ping scan”, and tells nmap to send an icmp echo request, TCP SYN to port 443, TCP ACK to port 80 and icmp timestamp request to all hosts in the specified subnet. nmap will simply return a list of ip’s that responded. Unlike many nmap commands this particular one does not require root privileges, however when executed by root nmap will also by default send arp requests to the subnet.

Scan for open ports (no root)

 $ nmap 192.168.0.0/24
 Starting Nmap 5.21 ( http://nmap.org ) at 2013-02-24 09:23 MST
 Nmap scan report for 192.168.0.1
 Host is up (0.0043s latency).
 Not shown: 998 closed ports
 PORT STATE SERVICE
 80/tcp open http
 443/tcp open https

This scan is the default scan for nmap and can take some time to generate. With this scan nmap will attempt a TCP SYN connection to 1000 of the most common ports as well as an icmp echo request to determine if a host is up. nmap will also perform a DNS reverse lookup on the identified ip’s as this can sometimes be useful information.

Identify the Operating System of a host (requires root)

 # nmap -O 192.168.0.164
 Starting Nmap 5.21 ( http://nmap.org ) at 2013-02-24 09:49 MST
 Nmap scan report for 192.168.0.164
 Host is up (0.00032s latency).
 Not shown: 996 closed ports
 PORT STATE SERVICE
 88/tcp open kerberos-sec
 139/tcp open netbios-ssn
 445/tcp open microsoft-ds
 631/tcp open ipp
 MAC Address: 00:00:00:00:00:00 (Unknown)
 Device type: general purpose
 Running: Apple Mac OS X 10.5.X
 OS details: Apple Mac OS X 10.5 - 10.6 (Leopard - Snow Leopard) (Darwin 9.0.0b5 - 10.0.0)
 Network Distance: 1 hop

With the -O option nmap will try to guess the targets operating system. This is accomplished by utilizing information that nmap is already getting through the TCP SYN port scan. This is usually a best guess but can actually be fairly accurate. The operating system scan however does require root privileges.

Identify Hostnames (no root)

 $ nmap -sL 192.168.0.0/24
 Starting Nmap 5.21 ( http://nmap.org ) at 2013-02-24 09:59 MST
 Nmap scan report for 192.168.0.0
 Nmap scan report for router.local (192.168.0.1)
 Nmap scan report for fakehost.local (192.168.0.2)
 Nmap scan report for another.fakehost.local (192.168.0.3)

This is one of the most subtle commands of nmap, the -sL flag tells nmap to do a simple DNS query for the specified ip. This allows you to find hostnames for all of the ip’s in a subnet without having send a packet to the individual hosts themselves.

Hostname information can tell you a lot more about a network than you would think, for instance if you labeled your Active Directory Servers with ads01.domain.com you shouldn’t be surprised if someone guesses its use.

TCP Syn and UDP Scan (requires root)

 # nmap -sS -sU -PN 192.168.0.164
 Starting Nmap 5.21 ( http://nmap.org ) at 2013-02-24 13:25 MST
 Nmap scan report for 192.168.0.164
 Host is up (0.00029s latency).
 Not shown: 1494 closed ports, 496 filtered ports
 PORT STATE SERVICE
 88/tcp open kerberos-sec
 139/tcp open netbios-ssn
 445/tcp open microsoft-ds
 631/tcp open ipp
 88/udp open|filtered kerberos-sec
 123/udp open ntp
 137/udp open netbios-ns
 138/udp open|filtered netbios-dgm
 631/udp open|filtered ipp
 5353/udp open zeroconf

The TCP SYN and UDP scan will take a while to generate but is fairly unobtrusive and stealthy. This command will check about 2000 common tcp and udp ports to see if they are responding. When you use the -Pn flag this tells nmap to skip the ping scan and assume the host is up. This can be useful when there is a firewall that might be preventing icmp replies.

TCP SYN and UDP scan for all ports (requires root)

 # nmap -sS -sU -PN -p 1-65535 192.168.0.164
 Starting Nmap 5.21 ( http://nmap.org ) at 2013-02-24 10:18 MST
 Nmap scan report for 192.168.0.164
 Host is up (0.00029s latency).
 Not shown: 131052 closed ports
 PORT STATE SERVICE
 88/tcp open kerberos-sec
 139/tcp open netbios-ssn
 445/tcp open microsoft-ds
 631/tcp open ipp
 17500/tcp open unknown
 88/udp open|filtered kerberos-sec
 123/udp open ntp
 137/udp open netbios-ns
 138/udp open|filtered netbios-dgm
 631/udp open|filtered ipp
 5353/udp open zeroconf
 17500/udp open|filtered unknown
 51657/udp open|filtered unknown
 54658/udp open|filtered unknown
 56128/udp open|filtered unknown
 57798/udp open|filtered unknown
 58488/udp open|filtered unknown
 60027/udp open|filtered unknown

This command is the same as above however by specifying the full port range from 1 to 65535 nmap will scan to see if the host is listening on all available ports. You can use the port range specification on any scan that performs a port scan.

TCP Connect Scan (no root)

 $ nmap -sT 192.168.0.164
 Starting Nmap 5.21 ( http://nmap.org ) at 2013-02-24 12:48 MST
 Nmap scan report for 192.168.0.164
 Host is up (0.0014s latency).
 Not shown: 964 closed ports, 32 filtered ports
 PORT STATE SERVICE
 88/tcp open kerberos-sec
 139/tcp open netbios-ssn
 445/tcp open microsoft-ds
 631/tcp open ipp

This command is similar to the TCP SYN scan however rather than sending a SYN packet and reviewing the headers it will ask the OS to establish a TCP connection to the 1000 common ports.

Aggressively Scan Hosts (no root)

 $ nmap -T4 -A 192.168.0.0/24
 Nmap scan report for 192.168.0.95
 Host is up (0.00060s latency).
 Not shown: 996 closed ports
 PORT STATE SERVICE VERSION
 22/tcp open ssh OpenSSH 5.9p1 Debian 5ubuntu1 (protocol 2.0)
 | ssh-hostkey: 1024 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:6c (DSA)
 |_2048 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:6c (RSA)
 80/tcp open http nginx 1.1.19
 |_http-title: 403 Forbidden
 |_http-methods: No Allow or Public header in OPTIONS response (status code 405)
 111/tcp open rpcbind
 | rpcinfo:
 | program version port/proto service
 | 100000 2,3,4 111/tcp rpcbind
 | 100000 2,3,4 111/udp rpcbind
 | 100003 2,3,4 2049/tcp nfs
 | 100003 2,3,4 2049/udp nfs
 | 100005 1,2,3 46448/tcp mountd
 | 100005 1,2,3 52408/udp mountd
 | 100021 1,3,4 35394/udp nlockmgr
 | 100021 1,3,4 57150/tcp nlockmgr
 | 100024 1 49363/tcp status
 | 100024 1 51515/udp status
 | 100227 2,3 2049/tcp nfs_acl
 |_ 100227 2,3 2049/udp nfs_acl
 2049/tcp open nfs (nfs V2-4) 2-4 (rpc #100003)
 Service Info: OS: Linux; CPE: cpe:/o:linux:kernel

Unlike some of the earlier commands this command is very aggressive and very obtrusive. The -A simply tells nmap to perform OS checking and version checking. The -T4 is for the speed template, these templates are what tells nmap how quickly to perform the scan. The speed template ranges from 0 for slow and stealthy to 5 for fast and obvious.

Fast Scan (no root)

 $ nmap -T4 -F 192.168.0.164
 Starting Nmap 6.01 ( http://nmap.org ) at 2013-02-24 12:49 MST
 Nmap scan report for 192.168.0.164
 Host is up (0.00047s latency).
 Not shown: 96 closed ports
 PORT STATE SERVICE
 88/tcp open kerberos-sec
 139/tcp open netbios-ssn
 445/tcp open microsoft-ds
 631/tcp open ipp

This scan limits the scan to the most common 100 ports, if you simply want to know some potential hosts with ports open that shouldn’t be this is a quick and dirty command to use.

Verbose

 $ nmap -T4 -A -v 192.168.0.164
 Starting Nmap 6.01 ( http://nmap.org ) at 2013-02-24 12:50 MST
 NSE: Loaded 93 scripts for scanning.
 NSE: Script Pre-scanning.
 Initiating Ping Scan at 12:50
 Scanning 192.168.0.164 [2 ports]
 Completed Ping Scan at 12:50, 0.00s elapsed (1 total hosts)
 Initiating Parallel DNS resolution of 1 host. at 12:50
 Completed Parallel DNS resolution of 1 host. at 12:50, 0.01s elapsed
 Initiating Connect Scan at 12:50
 Scanning 192.168.0.164 [1000 ports]
 Discovered open port 139/tcp on 192.168.0.164
 Discovered open port 445/tcp on 192.168.0.164
 Discovered open port 88/tcp on 192.168.0.164
 Discovered open port 631/tcp on 192.168.0.164
 Completed Connect Scan at 12:50, 5.22s elapsed (1000 total ports)
 Initiating Service scan at 12:50
 Scanning 4 services on 192.168.0.164
 Completed Service scan at 12:51, 11.00s elapsed (4 services on 1 host)
 NSE: Script scanning 192.168.0.164.
 Initiating NSE at 12:51
 Completed NSE at 12:51, 12.11s elapsed
 Nmap scan report for 192.168.0.164
 Host is up (0.00026s latency).
 Not shown: 996 closed ports
 PORT STATE SERVICE VERSION
 88/tcp open kerberos-sec Mac OS X kerberos-sec
 139/tcp open netbios-ssn Samba smbd 3.X (workgroup: WORKGROUP)
 445/tcp open netbios-ssn Samba smbd 3.X (workgroup: WORKGROUP)
 631/tcp open ipp CUPS 1.4
 | http-methods: GET HEAD OPTIONS POST PUT
 | Potentially risky methods: PUT
 |_See http://nmap.org/nsedoc/scripts/http-methods.html
 | http-robots.txt: 1 disallowed entry
 |_/
 Service Info: OS: Mac OS X; CPE: cpe:/o:apple:mac_os_x

By adding verbose to a majority of the commands above you get a better insight into what nmap is doing; for some scans verbosity will provide additional details that the report does not provide.

While these are 10 very useful nmap commands I am sure there are some more handy nmap examples out there. If you have one to add to this list feel free to drop it into a comment.

Source: http://bencane.com/2013/02/25/10-nmap-commands-every-sysadmin-should-know/

Netdata Custom Dashboards

You can:

  • create your own dashboards using simple HTML (no javascript is required for basic dashboards)
  • utilizing any or all of the available chart libraries, on the same dashboard
  • using data from one or more netdata servers, on the same dashboard
  • host your dashboard HTML page on any web server, anywhere

netdata charts can also be added to existing web pages.

Check this very simple working example of a custom dashboard, and its html source.

If you plan to put it on TV, check tv.html. This is a screenshot of it, monitoring 2 servers on the same page:

image

Web directory

The default web root directory is /usr/share/netdata/web where you will find examples such as tv.html, and demo.html as well as the main dashboard contained in index.html.
Note: index.html have a different syntax. Don’t use it as a template for simple custom dashboards.

Example empty dashboard

If you need to create a new dashboard on an empty page, we suggest the following header:

<!DOCTYPE html>
<html lang="en">
<head>
	<title>Your dashboard</title>

	<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
	<meta charset="utf-8">
	<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
	<meta name="viewport" content="width=device-width, initial-scale=1">
	<meta name="apple-mobile-web-app-capable" content="yes">
	<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">

	<!-- here we will add dashboard.js -->

</head>
<body>

<!-- here we will add charts -->

</body>
</html>

dashboard.js

To add netdata charts to any web page (dedicated to netdata or not), you need to include the /dashboard.js file of a netdata server.

For example, if your netdata server listens at http://box:19999/, you will need to add the following to the head section of your web page:

<script type="text/javascript" src="http://box:19999/dashboard.js"></script>

what dashboard.js does?

dashboard.js will automatically load the following:

  1. dashboard.css, required for the netdata charts
  2. jquery.min.js, (only if jquery is not already loaded for this web page)
  3. bootstrap.min.js (only if bootstrap is not already loaded) and bootstrap.min.css.You can disable this by adding the following before loading dashboard.js:
<script>var netdataNoBootstrap = true;</script>
  1. jquery.nanoscroller.min.js, required for the scrollbar of the chart legends.
  2. bootstrap-toggle.min.js and bootstrap-toggle.min.css, required for the settings toggle buttons.
  3. font-awesome.min.css, for icons.

When dashboard.js loads will scan the page for elements that define charts (see below) and immediately start refreshing them. Keep in mind more javascript modules may be loaded (every chart library is a different javascript file, that is loaded on first use).

Prevent dashboard.js from starting chart refreshes

If your web page is not static and you plan to add charts using javascript, you can tell dashboard.js not to start processing charts immediately after loaded, by adding this fragment before loading it:

<script>var netdataDontStart = true;</script>

The above, will inform the dashboard.js to load everything, but not process the web page until you tell it to. You can tell it to start processing the page, by running this javascript code:

NETDATA.start();

Be careful not to call the NETDATA.start() multiple times. Each call to this function will spawn a new thread that will start refreshing the charts.

If, after calling NETDATA.start() you need to update the page (or even get your javascript code synchronized with dashboard.js), you can call (after you loaded dashboard.js):

NETDATA.pause(function() {
  // ok, it is paused

  // update the DOM as you wish

  // and then call this to let the charts refresh:
  NETDATA.unpause();
});

The default netdata server

dashboard.js will attempt to auto-detect the URL of the netdata server it is loaded from, and set this server as the default netdata server for all charts.

If you need to set any other URL as the default netdata server for all charts that do not specify a netdata server, add this before loading dashboard.js:

<script type="text/javascript">var netdataServer = "http://your.netdata.server:19999";</script>

Adding charts

To add charts, you need to add a div for each of them. Each of these div elements accept a few data- attributes:

The chart unique ID

The unique ID of a chart is shown at the title of the chart of the default netdata dashboard. You can also find all the charts available at your netdata server with this URL: http://your.netdata.server:19999/api/v1/charts (example).

To specify the unique id, use this:

<div data-netdata="unique.id"></div>

The above is enough for adding a chart. It most probably have the wrong visual settings though. Keep reading…

The duration of the chart

You can specify the duration of the chart (how much time of data it will show) using:

<div data-netdata="unique.id"
     data-after="AFTER_SECONDS"
     data-before="BEFORE_SECONDS"
     ></div>

AFTER_SECONDS and BEFORE_SECONDS are numbers representing a time-frame in seconds.

The can be either:

  • absolute unix timestamps (in javascript terms, they are new Date().getTime() / 1000. Using absolute timestamps you can have a chart showing always the same time-frame.
  • relative number of seconds to now. To show the last 10 minutes of data, AFTER_SECONDS must be -600 (relative to now) and BEFORE_SECONDS must be 0 (meaning: now). If you want the chart to auto-refresh the current values, you need to specify relative values.

Chart dimensions

You can set the dimensions of the chart using this:

<div data-netdata="unique.id"
     data-width="WIDTH"
     data-height="HEIGHT"
     ></div>

WIDTH and HEIGHT can be anything CSS accepts for width and height (e.g. percentages, pixels, etc). Keep in mind that for certain chart libraries, dashboard.js may apply an aspect ratio to these.

If you want dashboard.js to remember permanently (browser local storage) the dimensions of the chart (the user may resize it), you can add: data-id="SETTINGS_ID", where SETTINGS_ID is anything that will be common for this chart across user sessions.

Netdata server

Each chart can get data from a different netdata server. You can give per chart the netdata server using:

<div data-netdata="unique.id"
     data-host="http://another.netdata.server:19999/"
     ></div>

Chart library

The default chart library is dygraph. You set a different chart library per chart using this:

<div data-netdata="unique.id"
     data-chart-library="gauge"
     ></div>

Each chart library may support more chart-library specific settings. Please refer to the documentation of the chart library you are interested, in this wiki.

Data points

For the time-frame requested, dashboard.js will use the chart dimensions and the settings of the chart library to find out how many data points it can show.

For example, most line chart libraries are using 3 pixels per data point. If the chart shows 10 minutes of data (600 seconds), its update frequency is 1 second, and the chart width is 1800 pixels, then dashboard.js will request from the netdata server: 10 minutes of data, represented in 600 points, and the chart will be refreshed per second. If the user resizes the window so that the chart becomes 600 pixels wide, then dashboard.js will request the same 10 minutes of data, represented in 200 points and the chart will be refreshed once every 3 seconds.

If you need to have a fixed number of points in the data source retrieved from the netdata server, you can set:

<div data-netdata="unique.id"
     data-points="DATA_POINTS"
     ></div>

Where DATA_POINTS is the number of points you need.

You can also overwrite the pixels-per-point per chart using this:

<div data-netdata="unique.id"
     data-pixels-per-point="PIXELS_PER_POINT"
     ></div>

Where PIXELS_PER_POINT is the number of pixels each data point should occupy.

Data grouping method

Netdata supports average (the default) or max grouping methods. The grouping method is used when the netdata server is requested to return fewer points for a time-frame, compared to the number of points available.

You can give it per chart, using:

<div data-netdata="unique.id"
     data-method="max"
     ></div>

Selecting dimensions

By default, dashboard.js will show all the dimensions of the chart. You can select specific dimensions using this:

<div data-netdata="unique.id"
     data-dimensions="dimension1,dimension2,dimension3,..."
     ></div>

Chart title

You can overwrite the title of the chart using this:

<div data-netdata="unique.id"
     data-title="my super chart"
     ></div>

Chart units

You can overwrite the units of measurement of the dimensions of the chart, using this:

<div data-netdata="unique.id"
     data-units="words/second"
     ></div>

Chart colors

dashboard.js has an internal palette of colors for the dimensions of the charts. You can prepend colors to it (so that your will be used first) using this:

<div data-netdata="unique.id"
     data-colors="#AABBCC #DDEEFF ..."
     ></div>

Extracting dimension values

dashboard.js can update the selected values of the chart at elements you specify. For example, let’s assume we have a chart that measures the bandwidth of eth0, with 2 dimensions in and out. You can use this:

<div data-netdata="net.eth0"
     data-show-value-of-in-at="eth0_in_value"
     data-show-value-of-out-at="eth0_out_value"
     ></div>

My eth0 interface, is receiving <span id="eth0_in_value"></span>
and transmitting <span id="eth0_out_value"></span>.

Hiding the legend of a chart

On charts that by default have a legend managed by dashboard.js you can remove it, using this:

<div data-netdata="unique.id"
     data-legend="no"
     ></div>

API options

You can append netdata REST API v1 data options, using this:

<div data-netdata="unique.id"
     data-append-options="absolute,percentage"
     ></div>

Chart library performance

dashboard.js measures the performance of the chart library when it renders the charts. You can specify an element ID you want this information to be visualized, using this:

<div data-netdata="unique.id"
     data-dt-element-name="measurement1"
     ></div>

refreshed in <span id="measurement1"></span> milliseconds!

https://github.com/firehol/netdata/wiki/Custom-Dashboards

Hyper-V IDE or SCSI? What’s Performing Better, Faster?

If you wonder whether to use IDE or SCSI controllers for your Hyper-V virtual machines, the short answer is: IDE is fine.

There is no need to go for SCSI, it won’t be any faster. Note that you need to have a IDE connected virtual disk in order to boot.

If you want better performance, the virtual machines will run much faster if you:

  1. Use pass through disks instead
  2. Use fixed sized VHDs
  3. Refrain from using snapshots / checkpoints
  4. Refrain from using dynamically expanding disks
  5. Have at least 15% free space inside the VM at all times, and at least 10GB free. It’s an old characteristic of NTFS….
  6. Use paging files on a separate VHD, ideally hosted on a separate drive
  7. Use fixed-sized paging files
  8. Use 4KB NTFS cluster size on the host

Yes, from the performance side, a VM with IDE drives needs less processing to emulate IDE than using SCSI. Otherwise, in my experience, I did not have any breaks in using SCSI on Windows machines over using IDE. However I must say that during synthetic benchmarks, SCSI seems to be a little faster than IDE on Hyper-V.

https://community.spiceworks.com/topic/451467-hyperv-scsi-controller-v-ide-controller

Add or Remove Physical Hard Disk for Hyper-V Virtual Machine

Hyper-V enables running virtualized computer systems on top of a physical host. These virtualized systems (aka: guests) can be used and managed just as if they were physical computer systems, however they exist in a virtualized and isolated environment.

Your Hyper-V virtual machines can also be connected to physical hard disks from the host computer—not just to virtual hard disks. (This is sometimes referred to as having a “pass-through” disk connected to a virtual machine.)

This tutorial will show you how to add and remove physical hard disks to access from a Hyper-V virtual machine in Windows 8 and Windows 10.

Note   Note
Hyper-V is only available in the Window 8 Pro, Windows 8 Enterprise, Windows 10 Pro, Windows 10 Enterprise, and Windows 10 Education editions.

You can add hard drives (ex: HDD and SSD) and removable USB hard drives, but you will not be able to add removable media (ex: USB flash drive) to a Hyper-V virtual machine.

While you have a physical hard disk added to a Hyper-V virtual machine, you will not be able to create a checkpoint for the virtual machine.
CONTENTS:

  • Option One: To Add Physical Hard Disk to Hyper-V Virtual Machine
  • Option Two: To Remove Physical Hard Disk from Hyper-V Virtual Machine

 

To Add Physical Hard Disk to Hyper-V Virtual Machine
1. Open Disk Management (diskmgmt.msc).

2. Right click on the online disk (ex: Disk 3 – “Internal HDD”) you want to add to the VM, and click/tap on Offline. (see screenshot below)

Note   Note
It’s required that the physical hard disk be in an offline state on the host computer to be able to add to the VM.

Click image for larger version. 

Name:	Add_drive_to_Hyper-V_virtual_machine-1.png 
Views:	82 
Size:	71.5 KB 
ID:	90147

3. Once the disk is offline, you can close Disk Management if you like.

4. Open the settings of the Hyper-V virtual machine you want to add the disk to. (see screenshots below)

Note   Note
It doesn’t matter if you currently have the virtual machine off or running.

Name:  Hyper-V_VM_settings-1.png
Views: 1860
Size:  21.4 KB
Click image for larger version. 

Name:	Hyper-V_VM_settings-2.jpg 
Views:	236 
Size:	148.3 KB 
ID:	90156

5. In the VM’s settings, click/tap on SCSI Controller in the left navigation pane, select Hard Drive on the right side, and click/tap on the Add button. (see screenshot below)

Name:  Add_drive_to_Hyper-V_virtual_machine-3.png
Views: 607
Size:  60.5 KB

6. Select (dot) Physical hard disk on the right side, select the disk you want to add in the drop down menu, and click/tap on OK. (see screenshot below)

Name:  Add_drive_to_Hyper-V_virtual_machine-4.png
Views: 581
Size:  68.5 KB

7. The disk will now be available to access in the virtual machine. (see screenshot below)

Click image for larger version. 

Name:	Add_drive_to_Hyper-V_virtual_machine-5.jpg 
Views:	74 
Size:	123.7 KB 
ID:	90150

To Remove Physical Hard Disk from Hyper-V Virtual Machine

1. Open the settings of the Hyper-V virtual machine you want to remove the disk from. (see screenshots below)

Note   Note
It doesn’t matter if you currently have the virtual machine off or running.

Name:  Hyper-V_VM_settings-1.png
Views: 1860
Size:  21.4 KB
Click image for larger version. 

Name:	Hyper-V_VM_settings-2.jpg 
Views:	236 
Size:	148.3 KB 
ID:	90156

2. In the VM’s settings, select the disk you want to remove under SCSI Controller in the left navigation pane, and click/tap on the Remove button on the right side. (see screenshot below)

Name:  Remove_drive_from_Hyper-V_virtual_machine-2.png
Views: 608
Size:  66.9 KB

3. click/tap on OK. (see screenshot below)

Name:  Remove_drive_from_Hyper-V_virtual_machine-3.png
Views: 582
Size:  61.2 KB

4. The disk will now be removed from the virtual machine. (see screenshot below)

Click image for larger version. 

Name:	Remove_drive_from_Hyper-V_virtual_machine-5.jpg 
Views:	50 
Size:	118.7 KB 
ID:	90154

5. Open Disk Management (diskmgmt.msc).

6. Right click on the offline disk (ex: Disk 3 – “Internal HDD”) you removed from the VM, and click/tap on Online. (see screenshot below)

Note   Note
You will need to set the disk back to an online state to be able to access it from your host computer again.

Click image for larger version. 

Name:	Remove_drive_from_Hyper-V_virtual_machine-4.png 
Views:	47 
Size:	66.2 KB 
ID:	90163

7. Once the disk is back online, you can close Disk Management if you like.

Source: https://www.tenforums.com/tutorials/56257-add-remove-physical-hard-disk-hyper-v-virtual-machine.html