10 Awesome Examples for Viewing Huge Log Files in Unix

Viewing huge log files for trouble shooting is a mundane routine tasks for sysadmins and programmers.

In this article, let us review how to effectively view and manipulate huge log files using 10 awesome examples.

 

Example 1: Display specific lines (based on line number) of a file using sed command

View only the specific lines mentioned by line numbers.

Syntax: $ sed -n -e Xp -e Yp FILENAME
  • sed : sed command, which will print all the lines by default.
  • -n : Suppresses output.
  • -e CMD : Command to be executed
  • Xp: Print line number X
  • Yp: Print line number Y
  • FILENAME : name of the file to be processed.

The example mentioned below will print the lines 120, 145, 1050 from the syslog.

$ sed -n -e 120p -e 145p -e 1050p /var/log/syslog

In the following example,  you can view the content of var/log/cron from line number 101 to 110.

  • M – Starting line number
  • N – Ending line number
Syntax: sed -n M,Np FILENAME

$ sed -n 101,110p /var/log/cron

Example 2: Display first N lines of a file using head command

This example displays only first 15 lines of /var/log/maillog file. Change 15 to 10 to display the first 10 lines of a log file.

Syntax: head -n N FILENAME

$ head -n 15 /var/log/maillog

Example 3: Ignore last N lines of a file using head command

This example shows how to ignore the last N lines, and show only the remaining lines from the top of file.

The following example will display all the lines of the /var/log/secure except the last 250 lines.

Syntax: head -n -N FILENAME

$ head -n -250 /var/log/secure

Example 4: Display last N lines of the file using tail command

This example displays only last 50 lines of /var/log/messages file. Change 50 to 100 to display the last 100 lines of the log file.

Syntax: tail -n N FILENAME

$ tail -n 50 /var/log/messages

Example 5: Ignore first N-1 lines of the file using tail command

This example shows how to ignore the first N-1 lines and show only the remaining of the lines.

The following example ignores the 1st four lines of the /etc/xinetd.conf, which contains only the comments.

Syntax: tail -n +N FILENAME

$ tail -n +5 /etc/xinetd.conf
defaults
{
        instances               = 60
        log_type                = SYSLOG authpriv
        log_on_success          = HOST PID
        log_on_failure          = HOST
        cps                     = 25 30
}
includedir /etc/xinetd.d

Example 6: View growing log file in real time using tail command

This is probably one of the most used command by sysadmins.To view a growing log file and see only the newer contents use tail -f as shown below.

The following example shows the content of the /var/log/syslog command in real-time.

Syntax: tail -f FILENAME

$ tail -f /var/log/syslog

Example 7: Display specific lines (based on line number) of a file using head and tail command

The example below will display line numbers 101 – 110 of /var/log/anaconda.log file

  • M – Starting line number
  • N – Ending line number
Syntax: cat file | tail -n +N | head -n (M-N+1)

$ cat /var/log/anaconda.log | tail -n +101 | head -n 10
  • cat : prints the whole file to the stdout.
  • tail -n +101 : ignores lines upto the given line number, and then start printing lines after the given number.
  • head -n 10 : prints the first 10 line, that is 101 to 110 and ignores the remaining lines.

Example 8: Display lines matching a pattern, and few lines following the match.

The following example displays the line that matches “Initializing CPU” from the /var/log/dmesg and 5 lines immediately after this match.

# grep "Initializing CPU#1" /var/log/dmesg
Initializing CPU#1
[Note: The above shows only the line matching the pattern]

# grep -A 5 "Initializing CPU#1" dmesg
Initializing CPU#1
Calibrating delay using timer specific routine.. 3989.96 BogoMIPS (lpj=1994982)
CPU: After generic identify, caps: bfebfbff 20100000 00000000 00000000
CPU: After vendor identify, caps:  bfebfbff 20100000 00000000 00000000
monitor/mwait feature present.
CPU: L1 I cache: 32K, L1 D cache: 32K
[Note: The above shows the line and 5 lines after the pattern matching]

Refer our earlier article Get a Grip on the Grep! – 15 Practical Grep Command Examples that explains how to use grep command.

As explained in our previous grep command article, the following operations are possible.

  • Viewing specific lines identified by patterns, which is grep’s default functionality.
  • Viewing only the matched characters.
  • Viewing N lines after the match with -A option.
  • Viewing N lines before the match with -B option.
  • Viewing N lines around the match with -C option.

Example 9: Displaying specific bytes from a file.

The following example explains how to display either the top 40 or the last 30 bytes of a file.

Display first 40 bytes from syslog.

$ head -c40 /var/log/syslog

Display last 30 bytes from syslog.

$ tail -c30 /var/log/syslog

Example 10: Viewing compressed log files

After a specific time all the system log files are rotated, and compressed. You can uncompress it on the fly, and pipe the output to another unix command to view the file as explained below.

Refer to our earlier article The Power of Z Commands – Zcat, Zless, Zgrep, Zdiff Examples

  • Display the first N lines of a compressed file.
    $ zcat file.gz | head -250
  • Display the last N lines of a compressed file.
    $ zcat file.gz | tail -250
  • Ignoring the last N lines of a compressed file.
    $ zcat file.gz | head -n -250
  • Ignoring the first N lines of a compressed file.
    $ zcat file.gz | tail -n +250
  • Viewing the lines matching the pattern
    $ zcat file.gz | grep -A2 'error'
  • Viewing particular range of lines identified by line number.
    $ zcat file.gz | sed -n -e 45p -e 52p

If you need to return, bookmark this page at del.icio.us for handy reference.

 

Source: http://www.thegeekstuff.com/2009/08/10-awesome-examples-for-viewing-huge-log-files-in-unix/

Advertisements

How to enable remote connections in SQL Server 2008?

How to enable remote connections in SQL Server 2008?

Source: http://blogs.msdn.com/b/walzenbach/archive/2010/04/14/how-to-enable-remote-connections-in-sql-server-2008.aspx

You experience the following error message:

A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 – Could not open a connection to SQL Server)
SQL Server 2008: The server was not found or was not accessible.

How to solve this issue?

There are a couple of things that might be going on here… (All of the following configurations are made on the computer running your SQL Server 2008 instance)

Allow remote connections to this server

The first thing you want to check is if Remote Connections are enabled on your SQL Server database. In SQL Server 2008 you do this by opening SQL Server 2008 Management Studio, connect to the server in question, right click the server…

SQL Server 2008: Server Properties

… and open the Server Properties.

SQL Server 2008: Server Properties - Connections

Navigate to Connections and ensure that Allow remote connections to this server is checked. Check if this solves the problem. If it does, here you go, continue with whatever you were doing and have a nice day.

Protocols for MSSQLServer

If you’re still running in issues let’s dig a bit deeper. The next good thing to check is the SQL Server Network Configuration. Open the SQL Server Configuration Manager, unfold the node SQL Server Network Configuration and select Protocols for MSSQLServer (or whatever the name of your SQL Server instance is).

SQL Server 2008: Protocols for MSSQLServer

Make sure that TCP/IP is enabled and try again. Even though I hope that this resolved your problems there might still be an issue with…

The Firewall

If there is still no communication happening between your computer and the remote SQL Server you most likely need to configure your firewall settings. A good first step is to figure out which port is being used by TCP/IP (and which you need to open in your firewall). You can do this by right clicking TCP/IP and selecting Properties.

SQL Server 2008: TCP/IP Properties

Click on the tab IP Addresses and voilà – Port 1433 it is 🙂 That was easy enough and all there is left to do is to allow inbound TCP/IP traffic on Port 1433 in your firewall. In Windows 7 this works something like this. Open the Control Panel and navigate to Windows Firewall.

Microsoft Windows 7 Firewall settings

Click on Advanced Settings on the left hand side and you should see the Windows Firewall with Advanced Security. Select the Inboud Rules on the left hand side and click on New Rule… on the right hand side.

Microsoft Windows 7 Firewall with Advanced Security

This opens the New Inbound Rule Wizard which you can use to allow inbound traffic on Port 1433 for TCP/IP (and which is exactly how you configured your SQL Server in the steps above). Just follow the steps outlined below and you should be good 🙂

New Inbound Rule Wizard - Protocols and Ports

New Inbound Rule Wizard - Protocols and Ports

New Inbound Rule Wizard - Action

New Inbound Rule Wizard - Profile

New Inbound Rule Wizard - Name

That’s it, success! From here you should be able to access your SQL Server remotely. Enjoy!

Cheers!

Daniel

P.S. There’s a good article on msdn on this topic if you’re looking for additional information >> Configuring the Windows Firewall to Allow SQL Server Access.

Update (June 14th, 2010):

,