Zimbra MTA

The Zimbra MTA (Mail Transfer Agent) receives mail via SMTP and routes each message, using Local Mail Transfer Protocol (LMTP), to the appropriate Zimbra mailbox server.

The Zimbra MTA server includes the following programs:

Postfix MTA, for mail routing, mail relay, and attachment blocking
Clam AntiVirus, an antivirus engine used for scanning email messages and attachments in email messages for viruses
SpamAssassin and DSPAM, mail filters that attempt to identify unsolicited commercial email (spam), using a variety of mechanisms
Amavisd-New, a Postfix content filter used as an interface between Postfix and ClamAV / SpamAssassin

In the Zimbra Collaboration Suite configuration, mail transfer and delivery are distinct functions. Postfix primarily acts as a Mail Transfer Agent (MTA) and the Zimbra mail server acts as a Mail Delivery agent (MDA).

MTA configuration is stored in LDAP and a configuration script automatically polls the LDAP directory every two minutes for modifications, and updates the Postfix configuration files with the changes.

Zimbra MTA Deployment

The Zimbra Collaboration Suite includes a precompiled version of Postfix. This version does not have any changes to the source code, but it does include configuration file modifications, additional scripts, and tools.

Postfix performs the Zimbra mail transfer and relay. It receives inbound messages via SMTP, and hands off the mail messages to the Zimbra server via LMTP, as shown in the following figure. The Zimbra MTA can also perform anti-virus and anti-spam filtering.

Postfix also plays a role in transfer of outbound messages. Messages composed from the Zimbra web client are sent by the Zimbra server through Postfix, including messages sent to other users on the same Zimbra server.

Figure 6: Postfix in a Zimbra Environment

6 MTA.5.1.1.jpg

*The term “edge MTA” is a generic term referring to any sort of edge security solution for mail. You may already deploy such solutions for functions such as filtering. The edge MTA is optional. Some filtering may be duplicated between an edge MTA and the Zimbra MTA.

Postfix Configuration Files

Zimbra modified the following Postfix files specifically to work with the Zimbra Collaboration Suite:

main.cf – Modified to include the LDAP tables. The configuration script in the Zimbra MTA pulls data from the Zimbra LDAP and modifies the Postfix configuration files.
master.cf – Modified to use Amavisd-New.

Important: Do not modify the Postfix configuration files directly! Some of the Postfix files are rewritten when changes are made in the administration console. Any changes you make will be overwritten.

MTA Functionality

Zimbra MTA Postfix functionality includes:

SMTP authentication
Attachment blocking
Relay host configuration
Postfix-LDAP integration
Integration with Amavisd-New, ClamAV, and Spam Assassin

SMTP Authentication

SMTP authentication allows authorized mail clients from external networks to relay messages through the Zimbra MTA. The user ID and password is sent to the MTA when the SMTP client sends mail so the MTA can verify if the user is allowed to relay mail.

Note: User authentication is provided through the Zimbra LDAP directory server, or if implemented, through the Microsoft Active Directory Sever.

SMTP Restrictions

In the administration console, you can enable restrictions so that messages are not accepted by Postfix when non-standard or other disapproved behavior is exhibited by an incoming SMTP client. These restrictions provide some protection against ill-behaved spam senders. By default, SMTP protocol violators (that is, clients that do not greet with a fully qualified domain name) are restricted. DNS based restrictions are also available.

Important: Understand the implications of these restrictions before you implement them. You may want to receive mail from people outside of your mail system, but those mail systems may be poorly implemented. You may have to compromise on these checks to accommodate them.

Relay Host Settings

Postfix can be configured to send all non-local mail to a different SMTP server. Such a destination SMTP server is commonly referred to as a “relay” or “smart” host. You can set this relay host from the administration console.

A common use case for a relay host is when an ISP requires that all your email be relayed through designated host, or if you have some filtering SMTP proxy server.

In the administration console, the relay host setting must not be confused with web mail MTA setting. Relay host is the MTA to which Postfix relays non-local email. Webmail MTA is used by the Zimbra server for composed messages and must be the location of the Postfix server in the Zimbra MTA package.

Important: Use caution when setting the relay host to prevent mail loops

MTA-LDAP Integration

The Zimbra LDAP directory service is used to look up email delivery addresses. The version of Postfix included with Zimbra is configured during the installation of the Zimbra Collaboration Suite to use the Zimbra LDAP directory.

Account Quota and the MTA

Account quota is the storage limit allowed for an account. Account quotas can be set by COS or per account. The MTA attempts to deliver a message, and if a Zimbra user’s mailbox exceeds the set quota, the Zimbra mailbox server rejects the message as mailbox is full and the sender gets a bounce message. You can view account quotas from the Administration Console, Monitoring Server Statistics section.

MTA and Amavisd-New Integration

The Amavisd-New utility is the interface between the Zimbra MTA and Clam AV and SpamAssassin scanners.

Anti-Virus Protection

Clam AntiVirus software is bundled with the Zimbra Collaboration Suite as the virus protection engine. The Clam anti-virus software is configured to block encrypted archives, to send notification to administrators when a virus has been found, and to send notification to recipients alerting that a mail message with a virus was not delivered.

The anti-virus protection is enabled during installation. You can also enable or disable virus checking from Global Settings on the administration console. By default, the Zimbra MTA checks every two hours for any new anti-virus updates from ClamAV.

Note: Updates are obtained via HTTP from the ClamAV website.

Anti-Spam Protection

SpamAssassin and DSPAM are spam filters bundled with ZCS. When ZCS is installed, spam training is automatically enabled to let users train spam filters when they move messages in and out of their junk folders.

The SpamAssassin default configuration for ZCS is as follows:

zimbraSpamKillPercent: Spaminess percentage beyond which a message is dropped. Default kill percent at 75%. Mail that is scored at 75% is considered spam and is not delivered. SpamAssassin score of 20 is considered 100%. 75% equates to a spam score of 15.
zimbraSpamTagPercent: Spaminess percentage beyond which a message is marked as spam. Default tag percent at 33%. Mail that is scored at 33% is considered spam and is delivered to the Junk folder. Since a SpamAssassin score of 20 equates to 100%, the zimbraSpamTagPercent would equate to a spam score of 6.6.

A Subject Prefix can be configured so messages considered as spam are identified in the subject line as tagged as spam. When a message is tagged as spam, the message is delivered to the recipient’s Junk folder.

You can change these settings from the administration console, Global Settings Anti-Spam tab.

Note: ZCS configures the spam filter to add 0.5 to the Spamassassin score if DSPAM marks the message as spam and deduct 0.1 if DSPAM does not label it as spam.

Anti-Spam Training Filters

When ZCS is installed, the automated spam training filter is enabled and two feedback mailboxes are created to receive mail notification.

Spam Training User to receive mail notification about mail that was not marked as junk, but should be.
Non-spam (HAM) training user to receive mail notification about mail that was marked as junk, but should not have been.

For these training accounts, the mailbox quota is disabled (i.e. set to 0) and attachment indexing is disabled. Disabling quotas prevents bouncing messages when the mailbox is full.

How well the anti-spam filter works depends on recognizing what is considered spam or not considered spam. The SpamAssassin filter can learn what is spam and what is not spam from messages that users specifically mark as Junk from their web client toolbar or Not Junk from the web client Junk folder. A copy of these marked messages is sent to the appropriate spam training mailbox.The Zimbra spam training tool, zmtrainsa, is configured to automatically retrieve these messages and train the spam filter.

The zmtrainsa script is enabled through a cron job to feed mail that has been classified as spam or as non-spam to the SpamAssassin application, allowing SpamAssassin to ‘learn’ what signs are likely to mean spam or ham. The zmtrainsa script empties these mailboxes each day.

By default all users can give feedback in this way. If you do not want users to train the spam filter, you can modify the global configuration attributes, zimbraSpamIsSpamAccount and zimbraSpamIsNotSpamAccount, and remove the spam/ham account addresses from the attributes. To remove, type as:

zmprov mcf <attribute> ‘’

Restart the Zimbra services, type zmcontrol stop and then zmcontrol start.

When these attributes are modified, messages marked as junk or not junk are not copied to the spam training mailboxes.

Initially, you may want to train the spam filter manually to quickly build a database of spam and non-spam tokens, words, or short character sequences that are commonly found in spam or ham. To do this, you can manually forward messages as message/rfc822 attachments to the spam and non-spam mailboxes. When zmtrainsa runs, these messages are used to teach the spam filter. Make sure you add a large enough sampling of messages to these mailboxes. In order to get accurate scores to determine whether to mark messages as spam at least 200 known spams and 200 known hams must be identified.

The zmtrainsa command can be run manually to forward any folder from any mailbox to the spam training mailboxes. To send a folder to the spam training mailbox, type the command as:

zmtrainsa <server> <user> <password> spam [foldername]

To send the to the non-spam training mailbox, type:

zmtrainsa <server> <user> <password> ham [foldername]

Password is not needed in 4.5.6+ see CLI_zmtrainsa

Turning On or Off RBLs

See Customizing the MTA for current information

Receiving and Sending Mail through Zimbra MTA

The Zimbra MTA delivers both the incoming and the outgoing mail messages. For outgoing mail, the Zimbra MTA determines the destination of the recipient address. If the destination host is local, the message is passed to the Zimbra server for delivery. If the destination host is a remote mail server, the Zimbra MTA must establish a communication method to transfer the message to the remote host. For incoming messages, the MTA must be able to accept connection requests from remote mail servers and receive messages for the local users.

In order to send and receive email, the Zimbra MTA must be configured in DNS with both an [B_app-glossary.16.1.html#1037278 A record] and a [B_app-glossary.16.1.html#1021370 MX Record]. For sending mail, the MTA use DNS to resolve hostnames and email-routing information. To receive mail, the MX record must be configured correctly to route messages to the mail server.

You must configure a relay host if you do not enable DNS. Even if a relay host is configured, an MX record is still required if the server is going to receive email from the internet.

Zimbra MTA Message Queues

When the Zimbra MTA receives mail, it routes the mail through a series of queues to manage delivery. The Zimbra MTA maintains four queues where mail is temporarily placed while being processed: incoming, active, deferred and hold.

6 MTA.5.1.2.jpg

Incoming.

The incoming message queue holds the new mail that has been received. Each message is identified with a unique file name. Messages in the incoming queue are moved to the active queue when there is room in the active queue. If there are no problems, message move through this queue very quickly.

Active.

The active message queue holds messages that are ready to be sent. The MTA sets a limit to the number of messages that can be in the active queue at any one time. From here, messages are moved to and from the anti-virus and anti-spam filters before being delivered or moved to another queue.

Deferred.

Message that cannot be delivered for some reason are placed in the deferred queue. The reasons for the delivery failures is documented in a file in the deferred queue. This queue is scanned frequently to resend the message. If the message cannot be sent after the set number of delivery attempts, the message fails. The message is bounced back to the original sender.

Verified Against: Zimbra Collaboration 8.0, 7.0 Date Created: 04/16/2014
Article ID: https://wiki.zimbra.com/index.php?title=Zimbra_MTA Date Modified: 07/13/2015
Advertisements

60+ Free SysAdmin tools: Free IT software for business in 2016

IT professionals keep technology running smoothly at companies around the world so everyone else can do their job. But often, these hard-working, computer-savvy professionals are tasked with fixing everything and anything electronic, with little or no budget for software.

Thankfully, some helpful IT tools are available for free. But not all no-cost IT software is created equal. With freeware tools, sometimes you get what you pay for and busy IT workers don’t have time to individually test every application to separate the good apps from the not so good.

Great free tools for IT professionals

Often, the best way to find out about free IT tools and utilities is from others who have hands-on experience. And over the years, IT pros in Spiceworks have recommended many high-quality, free software tools to each other. We combed through many “best of” posts and reviews in Spiceworks to create this list of IT tools that are either completely free for commercial use or provide a free version that can serve a legitimate business purpose with an option to upgrade if more functionality is needed.

Free Backup Software

If you’re looking for low-cost or no-cost enterprise backup solutions for PCs and servers, check out these 5 free backup tools for IT pros. Most are completely free, while others are free versions of more powerful paid products, with a few small limitations.

  • Veeam Endpoint Backup Free — free for unlimited Windows-based desktops and laptops, this software can help anyone back up PCs to external hard drives or networked storage. (reviews)
  • Veeam Backup Free Edition — highly-rated virtualization backup software for creating ad-hoc backups of VMware or Hyper-V virtual machines (reviews)
  • Bacula — a network-based open source solution that can help sysadmins back up, recover, and verify data. Bacula claims they are the most popular open source backup program. (reviews)
  • Cobain Backup — freeware that can help you schedule incremental backups of files and directories (reviews)
  • Unitrends Enterprise Backup Free — free server backup virtual appliance that can help you protect your small business or home environment for up to 1TB of data (reviews)

Free Antivirus for Businesses

Free Text Editor

  • Notepad++ — a highly-rated free text editor for Windows that supports syntax highlighting for many file formats that’s available under the GNU General Public License (GPL), meaning it’s free to use by anyone, including businesses. (reviews)

Free Boot CD and Utilities

If your operating system won’t boot due to hard disk problems or a corrupted master boot record (MBR), boot discs or bootable USB sticks can help you recover files or figure out what’s wrong.

  • Ultimate Boot CD — a “completely free” boot disc that includes many diagnostic, disk, boot management, and benchmarking tools. Many IT pros consider it an essential tool in the IT pro arsenal. (reviews)
  • Knoppix — a fully functional Linux distribution / live CD that’s bootable from a USB drive, making it useful if you need to rescue data (reviews)
  • Hiren’s Boot CD — another bootable CD that includes many tools for data recovery, disk imaging, and general computer troubleshooting (reviews)

Free Disk Utilities

Whether you need to image a disk, permanently delete files, or blow away an entire hard drive, these free tools have you covered.

  • DBAN (Darik’s Boot And Nuke) — DBAN is one of the best wipe tools for hard drive disposal. If you need certification of erasure for compliance reporting, however, you should upgrade to the paid version
  • File Shredder — a free utility that can help you permanently delete Windows files and folders without leaving any trace, so they can’t be recovered later. (review)
  • Disk Wipe — a free Windows application for permanent data destruction that is “free for personal or commercual use, without any restrictions.” (review)
  • CloneZilla — a bootable disk imaging and cloning utility similar to other paid tools. Best of all, Clonezilla is free and open source. (review)
  • IMGBurn — a lightweight application that can help you burn CDs, DVDs, HD DVDs, or Blu-rays for free. (review)
  • FOG Computer Cloner — a Linux-based, free and open-source computer imaging solution for Windows that works via TFTP and PXE. No boot disks required, you can deploy images to multiple systems over the network. (review)

Free File Utilities 

  • 7zip — file compression software that’s free and open source and can be used on any computer in a commercial organization (reviews)
  • FreeFileSync — a GUI-based, rule-based file syncing tool that can help you perform complex sync operations. The tool is open-source and works on Windows, Linux, and Mac OS X folders. (reviews)
  • WinDirStat — free software that provides disk usage statistics and cleanup tools for Microsoft Windows so you can see which files and directories are hogging disk space and do something about it (reviews)

Free File Transfer Utilities 

  • Filezilla — a popular, open source FTP client and FTP server that’s free, open source, and distributed under the GNU General Public License (reviews)
  • Martin Prikryl WinSCP — an open source SFTP client and FTP client for Windows known for it’s distinct lack of bloatware that can facilitate secure file transfers between computers (reviews)

Free Virtualization Software

  • Oracle VM VirtualBox — a very useful open-source (GNU license for the base package) hypervisor that runs on top of your existing OS. VirtualBox runs on Windows, OS X, Linux, and Solaris hosts and supports a huge range of guest OSes including major Windows releases, Linux, OS/2, OpenBSD, DOS, OSX, and more. While free, additional features in the optional extension pack need to be licensed for commercial use. (reviews)
  • Disk2VHD — a freeware tool that helps you do physical to virtual conversions (P2V)  by generating a Virtual Hard Disk (VHD) file for use with . (reviews)
  • vCenter Converter — freeware that allows you to create VMware virtual machines by converting from Windows or Linux physical machines, or other VM formats. (reviews)

Microsoft Management Tools

  • Windows SysInternals — a collection of advanced system utilities for designed to help IT professionals manage, troubleshoot, and diagnose their Windows systems. (Reviews for AutorunsProcess Monitor, BgInfoActive Directory Explorer)
  • PowerShell — Microsoft’s highly-rated and powerful, .NET-based command line scripting and automation framework allows you to automate numerous processes and simplify systems management, saving sysadmins lots of valuable time. (reviews)

Free Network Monitoring and Management Tools

  • Connectivity Dashboard — monitor and view speed to their applications. Figure out if there’s an issue with the ISP, application, or something else onsite with this network troubleshooting tool. (review)
  • PDQ Deploy — this time-saving tool helps you deploy patches and common applications to multiple systems across your network. According to IT pros, the free version provides plenty of features if you don’t want to upgrade to the pro version (review)
  • The Dude — free, cross-platform network monitor application that can scan and map your network and alert you if there are any problems (reviews)
  • Wireshark — an open-source, multi-platform packet analyzer tool you can use to scan network traffic. (reviews)
  • Spiceworks Network Monitor — free, Windows-based monitoring software that provides real-time status and alerts for your critical devices including servers, switches, SNMP devices, services, and more. (reviews)
  • Nagios — free, cross-platform, open source software for network, systems, and infrastructure monitoring with built-in alerting. (reviews)
  • Cacti — an open-source, multi-tenant network monitor and graphing tool for Unix or Windows. It uses industry standards like SNMP to poll and graph info on CPU, bandwidth utilization, memory and more. (reviews)
  • Zabbix — an open-source, Linux-based, real-time network monitoring system for various network services, servers, and hardware. It can monitor up to 100,000 devices and provide up to 1,000,000 metrics. (reviews)
  • PRTG — an all-in-one monitoring solution for Windows, which continuously collects status information from IT infrastructure and informs you about malfunctions and allows you to pro-actively reduce downtime. The free version is limited to monitoring 100 sensors / switch ports. (reviews)

Free Software Firewall + IDS / IPS

Numerous free, software-based firewalls can help protect your business from external threats and some can even analyze incoming traffic for intrusions.

  • pfSense —   a free, open-source, FreeBSD-based software firewall distribution that can also be deployed as a router, wireless access point, DHCP server, DNS server, or VPN endpoint and with integrations, can also serve as an IDS or IPS solution. (reviews)
  • Untangle — a software-based next generation firewall platform with an intuitive GUI interface that’s popular among IT professionals. The free version includes firewall, IPS, and some web filtering functionality, although you can pay to unlock additional features (reviews)
  • Snort — free and open source network intrusion detection and prevention software used by many IT departments to secure their network (reviews)
  • IPCop — A Linux-based, open source firewall that’s geared towards small office and is a favorite among some IT pros (reviews)
  • Smoothwall Express — A Linux-based open source firewall solution that’s been around since 2000, and has an easy-to-use web interva (reviews)

Free Network Scanners

  • Advanced IP Scanner  — Windows freeware that can locate all computers on your network quickly and allows for some remote control of devices via RDP and Radmin (reviews)
  • Angry IP Scanner — a cross-platform, open source network scanner that can help you quickly find devices on your network. Available for Linux, Windows, and Mac OS X (reviews)
  • Spiceworks Inventory  — comprehensive network management freeware for Windows that provides a one-stop shop for cataloging devices, customized reporting and alerting, and software tracking (reviews)
  • NMAP — a cross-platform, open source network security scanner that can help you create a map of your network. Nmap can scan for exposed ports and services, which can help you identify vulnerabilities and make your network more secure. (reviews)

Free Help Desk Software

  • Spiceworks Cloud Help Desk — help desk software with nothing to install. Build, a multi-user help desk with no limits so you can tackle tickets and solve IT problems quickly, and get back on with your day.

Free Remote Desktop / Remote Control Software

    • Microsoft Remote Desktop Protocol (RDP) — included in most Server, Pro, and Enterprise versions of Windows for decades, RDP allows IT pros to remotely control computers from afar, although it doesn’t include some of the screen mirroring functionality that third-party remote desktop assistance software does. (reviews)
    • VNC variants — freeware that allows you to mirror someone else’s screen on yours remotely and control the mouse and keyboard too. Very useful for IT professionals who need to manage and debug many PCs.• Tight VNC — available for Windows and Unix (reviews)
      • UltraVNC — available for Windows (reviews)

 

  • Remote Desktop Manager (Devolutions) — A cross-platform remote desktop tool that works on Windows, OS X, Android, and iOS and supports multiple session types including RDP, VNC, Apple Remote Desktop, TeamViewer, and LogMeIn. A completely free version is available, with additional connectivity, password management, security, and document management features unlocked if you upgrade to the paid Enterprise edition. (reviews)

Free Office Productivity Software

  • LibreOffice — LibreOffice is free, open-source office suite software for Windows, Linux, OS X. It includes software for word processing, spreadsheets, presentations, vector graphics and flowcharts, databases, and formula editing and is a branch of the OpenOffice project. (reviews)
  • Apache OpenOffice — An open source, multi-platform office suite that includes word processing, spreadsheets, slideshow presentations, and more. It can be downloaded and used completely free of charge for any purpose. (reviews)

Free Web-based Network Tools from Spiceworks

  • Subnet Calculator —  does the difficult math of subnetting for you, so you can more easily divide an IP network into smaller subnets, displaying subnet masks, IP ranges, and CIDR notations graphically.
  • Port Scanner and Tester — Quickly find out which ports at an IP address or hostname are exposed to the internet
  • IP Lookup — Learn more about an unknown IP address or hostname like the ISP’s domain, organization, owner, and location.
  • Blacklist Check and IP Reputation — Check if an IP address or domain is on an email or website blacklist, so you an protect your network and users from threats.
  • Website Down Checker — Check if a website is down for just you or if it’s down for everyone. Useful for getting closer to the root cause of an outage.

Web-based Speed Tests

  • Fast.com — a super fast way to check download speed in terms of megabits per second (Mbps)
  • Speedtest.net — test ping, download, and upload speed using this free online site

Free Photo and Video Editors

  • GIMP — The GNU Image Manipulation Program is a powerful open source, cross platform (Windows, Linux, OS X) image editor similar to Photoshop that is free to use for businesses. (reviews)
  • Camstudio — screen recording software for Windows that’s “completely 100% free for your personal and commercial projects.”
  • Windows Movie Maker — Free video editing software for Windows from Microsoft that allows you to splice videos, add text, transitions, voice-overs, and more. (reviews)

Source: https://community.spiceworks.com/networking/articles/2511-60-free-sysadmin-tools-free-it-software-for-business-in-2016

Microsoft Power Bi – Add image, text, video, and more to your dashboard

Add tile

The Add tile control lets you directly add an image, text box, video, streaming data, or web code to your dashboard.

  1. Select Add tile from the top menu bar. Depending on space limitations, you may see only the plus  sign.

  2. Select which type of tile to add: ImageText boxVideoWeb content, or Custom streaming data.

Add an image

Say you want your company logo on your dashboard, or some other image. You’ll need to save the image file online and link to it. Make sure special credentials aren’t required to access the image file. For example, OneDrive and SharePoint require authentication, so images stored there can’t be added to a dashboard this way.

  1. Select Image > Next.

  2. Add image information to the Tile details pane.

    • to display a title above the image, select Display title and subtitle and type a title and/or subtitle.
    • enter the image URL
    • to make the tile a hyperlink, select Set custom link and enter the URL. When colleagues click this image or title, they’ll be taken to this URL.
    • Select Apply. On the dashboard, resize and move the image as needed.

Add a text box or dashboard heading

  1. Select Text box > Next.

    NOTE: To add a dashboard heading, type your heading in the text box and increase the font.

  2. Format the text box:

    • to display a title above the text box, select Display title and subtitle and type a title and/or subtitle.
    • enter and format content for the text box.
    • Optionally, set a custom link for the title. However, in this example we’ve added hyperlinks within the text box itself, so leave Set custom link unchecked.
  3. Select Apply. On the dashboard, resize and move the text box as needed.

Add a video

When you add a YouTube or Vimeo video tile to your dashboard, the video plays right on your dashboard.

  1. Select Video > Next.

  2. Add video information to the Tile details pane.

    • to display a title and subtitle at the top of the video tile, select Display title and subtitle and type a title and/or subtitle. In this example, we’ll add a subtitle and then turn it into a hyperlink back to the entire playlist on YouTube.
    • enter the URL for the video
    • Add a hyperlink for the title and subtitle. Perhaps after your colleagues watch the embedded video you’d like them to view the entire playlist on YouTube — add a link to your playlist here.
    • Select Apply. On the dashboard, resize and move the video tile as needed.

  3. Select the video tile to play the video.

  4. Select the subtitle to visit the playlist on YouTube.

Add streaming data

Add web content

Paste or type in any HTML content. Power BI adds it, as a tile, to your dashboard. Enter the embed code by hand or copy/paste from sites such as Twitter, YouTube, embed.ly, and more.

  1. Select Web content > Next.

  2. Add information to the Add web content tile pane.

    • to display a title above the tile, select Display title and subtitle and type a title and/or subtitle.
    • enter the embed code. In this example we’re copying and pasting a Twitter feed.
    • Select Apply. On the dashboard, resize and move the web content tile as needed.

Tips for embedding web content

  • For iframes, use a secure source. If you enter your iframe embed code and get a blank tile, check to see if you’re using http for the iframe source. If so, change it to https.
  https://xyz.com

If you’d like the player to resize to fit the tile size, set width and height to 100%.

  
  • This code embeds a tweet and retains, as separate links on the dashboard, links for the AFK podcast, @GuyInACube’s Twitter pageFollow#analyticsreplyretweet, and like. Selecting the tile itself takes you to the podcast on Twitter.
  <blockquote class="twitter-tweet" data-partner="tweetdeck">
  <p lang="en" dir="ltr">Listen to
  <a href="https://twitter.com/GuyInACube">@GuyInACube</a> talk to
  us about making videos about Microsoft Business Intelligence
  platform
  <a href="https://t.co/TmRgalz7tv">https://t.co/TmRgalz7tv </a>
  <a href="https://twitter.com/hashtag/analytics?src=hash">
  #analytics</a></p>&mdash; AFTK Podcast (@aftkpodcast) <a
  href="https://twitter.com/aftkpodcast/status/693465456531771392">
  January 30, 2016</a></blockquote> //platform.twitter.com/widgets.js

Edit a tile

To make changes to a tile…

  1. Hover over the top right corner of the tile and select the ellipses.

  2. Select the edit icon to re-open the Tile details pane and make changes.

Considerations and troubleshooting

  • To make it easier to move the tile on your dashboard, add a title and/or subtitle.

  • If you’d like to embed some content from a website, but the website doesn’t give you embed code to copy and paste, check out embed.ly for help generating the embed code.

Source: https://powerbi.microsoft.com/en-us/documentation/powerbi-service-add-a-widget-to-a-dashboard/

5 advantages of containers for writing applications

Containers can serve as a new way to package and architect applications. Pair them with DevOps and get ready for speed

Even Match.com could not have done a better job finding a mate for microservices. Microservices – single-function services built by small teams, independent from other functions, and communicating only through public interfaces – simply make a great match for containers. Microservices plus containers represent a shift to delivering applications through modular services that can be reused and rewired to perform new tasks.

Why do containers and writing apps go together so well?

Containerizing services like messaging, mobile app development and support, and integration lets developers build applications, integrate with other systems, orchestrate using rules and processes, and then deploy across hybrid environments.

[ How are your peers using containers – and what speed bumps should you avoid? See our related article, 4 container adoption patterns: What you need to know. ]

But don’t think of this as merely putting middleware into the cloud in its traditional form. Think of it as reimagining enterprise app development for faster, easier, and less error-prone provisioning and configuration. That adds up to more productive – and hopefully, less stressed – developers, especially at a time when speed is a core requirement for business.

When apps meet containers

One key idea behind microservices: Instead of large monolithic applications, application design will increasingly use architectures composed of small, single-function, independent services that communicate through network interfaces. This suits agile and DevOps approaches, and reduces the unintended effects associated with making changes in one part of a large monolithic program.

Linux containers can technically encapsulate monolithic applications effectively, just as if they were in a virtual machine or on a “bare metal” physical server. However, modern standards-compliant Linux container technology encourages breaking down applications into their separate processes and provides the tools to do so. (The Open Container Initiative – OCI – maintains standard runtime and image specifications for containers.)

This granular approach has several advantages:

1. Modularity equals flexibility

The current approach to containerization emphasizes the ability to update, restart, and scale components of an application independently – without unnecessarily taking down the whole app. In addition to this microservices-based approach, you can share functionality among multiple apps in much the same manner as service-oriented architectures more broadly. This means you’re not rewriting common functions (often in subtly incompatible ways) for every application.

2. Layers and image version control: DevOps win

Each container image file is made up of a series of layers. When the image changes, a new layer is created that’s essentially a set of filesystem changes. Configuration metadata such as environment variables or default arguments are properties of the image as a whole rather than any particular layer.

A variety of projects can be used to create images. These include the upstream Docker project, which requires a Dockerfile and a runtime daemon, while Buildah from Project Atomic can build a container from scratch.

The image layers are reused when building a new container image. This makes the build process fast and has tremendous advantages for organizations applying DevOps practices like continuous integration and deployment (CI/CD). Intermediate changes are shared between images, further improving speed, size, and efficiency. Inherent to layering is version control. Every time there’s a new change, you essentially get a built-in change-log.

3. Rollback: Fail fast safely

Perhaps the best part about layering is the ability to roll back. Every image has layers. Don’t like the current iteration of an image? Roll it back to the previous version. This further supports an agile development approach and helps make CI/CD a reality from a tools perspective.

4. Rapid deployment: Precious time gains

Getting new hardware up, running, provisioned, and available used to take days. And the level of effort and overhead was burdensome. OCI-compliant containers can reduce deployment to seconds. By creating a container for each process, developers can quickly share those similar processes with new apps.

And because an operating system doesn’t need to restart in order to add or move a container, deployment times are substantially shorter.

Think of technology as being in support of a more granular, controllable, microservices-oriented approach that places greater value on efficiency.

5. Orchestration: Take it to the next level

An OCI-compliant container runtime by itself is very good at managing single containers. However, when you start using more and more containers and containerized apps, broken down into hundreds of pieces, management and orchestration gets tricky. Eventually, you need to take a step back and group containers to deliver services – such as networking, security, and telemetry – across your containers.

Furthermore, because containers are portable, it’s important that the management stack that’s associated with them be portable as well. That’s where orchestration technologies like Kubernetes come in, simplifying this need for IT.

Rethinking applications

While containers can be used simply to encapsulate and isolate applications in a similar manner to virtual machines, they’re most effective when used as a fundamentally new way of packaging and architecting applications. Do this and pair them up with more agile and iterative DevOps processes, and you get apps that are more flexible, more reusable, and delivered more quickly.

For much more on containers and how they rewrite ideas about software packaging and development process, get my new book, which I wrote with my colleague William Henry: From Pots and Vats to Programs and Apps, freely downloadable at https://goo.gl/FSfgky.

Source: https://enterprisersproject.com/article/2017/8/5-advantages-containers-writing-applications

What is Kubernetes?

Kubernetes, or k8s (k, 8 characters, s…get it?), or “kube” if you’re into brevity, is an open source platform that automates Linux container operations. It eliminates many of the manual processes involved in deploying and scaling containerized applications. In other words, you can cluster together groups of hosts running Linux containers, and Kubernetes helps you easily and efficiently manage those clusters. These clusters can span hosts across publicprivate, or hybrid clouds.

Kubernetes was originally developed and designed by engineers at Google. Google was one of the early contributors to Linux container technology and has talked publicly about how everything at Google runs in containers. (This is the technology behind Google’s cloud services.) Google generates more than 2 billion container deployments a week—all powered by an internal platform: Borg. Borg was the predecessor to Kubernetes and the lessons learned from developing Borg over the years became the primary influence behind much of the Kubernetes technology.

Fun fact: The seven spokes in the Kubernetes logo refer to the project’s original name, “Project Seven of Nine.”

Red Hat was one of the first companies to work with Google on Kubernetes, even prior to launch, and has become the 2nd leading contributor to Kubernetes upstream project. Google donated the Kubernetes project to the newly formed Cloud Native Computing Foundation in 2015.


Why do you need Kubernetes?

Real production apps span multiple containers. Those containers must be deployed across multiple server hosts. Kubernetes gives you the orchestration and management capabilities required to deploy containers, at scale, for these workloads. Kubernetes orchestration allows you to build application services that span multiple containers, schedule those containers across a cluster, scale those containers, and manage the health of those containers over time.

Kubernetes also needs to integrate with networking, storage, security, telemetry and other services to provide a comprehensive container infrastructure.

Kubernetes explained - diagram

Of course, this depends on how you’re using containers in your environment. A rudimentary application of Linux containers treats them as efficient, fast virtual machines. Once you scale this to a production environment and multiple applications, it’s clear that you need multiple, colocated containers working together to deliver the individual services. This significantly multiplies the number of containers in your environment and as those containers accumulate, the complexity also grows.

Kubernetes fixes a lot of common problems with container proliferation—sorting containers together into a ”pod.” Pods add a layer of abstraction to grouped containers, which helps you schedule workloads and provide necessary services—like networking and storage—to those containers. Other parts of Kubernetes help you load balance across these pods and ensure you have the right number of containers running to support your workloads.

With the right implementation of Kubernetes—and with the help of other open source projects like Atomic RegistryOpen vSwitchheapsterOAuth, and SELinux— you can orchestrate all parts of your container infrastructure.


What can you do with Kubernetes?

The primary advantage of using Kubernetes in your environment is that it gives you the platform to schedule and run containers on clusters of physical or virtual machines. More broadly, it helps you fully implement and rely on a container-based infrastructure in production environments. And because Kubernetes is all about automation of operational tasks, you can do many of the same things that other application platforms or management systems let you do, but for your containers.

With Kubernetes you can:

  • Orchestrate containers across multiple hosts.
  • Make better use of hardware to maximize resources needed to run your enterprise apps.
  • Control and automate application deployments and updates.
  • Mount and add storage to run stateful apps.
  • Scale containerized applications and their resources on the fly.
  • Declaratively manage services, which guarantees the deployed applications are always running how you deployed them.
  • Health-check and self-heal your apps with autoplacement, autorestart, autoreplication, and autoscaling.

Kubernetes, however, relies on other projects to fully provide these orchestrated services. With the addition of other open source projects, you can fully realize the power of Kubernetes. These necessary pieces include (among others):

  • Registry, through projects like Atomic Registry or Docker Registry.
  • Networking, through projects like OpenvSwitch and intelligent edge routing.
  • Telemetry, through projects such as heapster, kibana, hawkular, and elastic.
  • Security, through projects like LDAP, SELinux, RBAC, and OAUTH with multi-tenancy layers.
  • Automation, with the addition of Ansible playbooks for installation and cluster life-cycle management.
  • Services, through a rich catalog of precreated content of popular app patterns.

Learn to speak Kubernetes

Like any technology, there are a lot of words specific to the technology that can be a barrier to entry. Let’s break down some of the more common terms to help you understand Kubernetes.

Master: The machine that controls Kubernetes nodes. This is where all task assignments originate.

Node: These machines perform the requested, assigned tasks. The Kubernetes master controls them.

Pod: A group of one or more containers deployed to a single node. All containers in a pod share an IP address, IPC, hostname, and other resources. Pods abstract network and storage away from the underlying container. This lets you move containers around the cluster more easily.

Replication controller:  This controls how many identical copies of a pod should be running somewhere on the cluster.

Service: This decouples work definitions from the pods. Kubernetes service proxies automatically get service requests to the right pod—no matter where it moves to in the cluster or even if it’s been replaced.

Kubelet: This service runs on nodes and reads the container manifests and ensures the defined containers are started and running.

kubectl: This is the command line configuration tool for Kubernetes.


Using Kubernetes in production

Kubernetes is open source. And, as such, there’s not a formalized support structure around that technology—at least not one you’d trust your business on. If you had an issue with your implementation of Kubernetes, while running in production, you’re not going to be very happy. And your customers probably won’t, either.

That’s where Red Hat OpenShift comes in. OpenShift is Kubernetes for the enterprise—and a lot more. OpenShift includes all of the extra pieces of technology that makes Kubernetes powerful and viable for the enterprise, including: registry, networking, telemetry, security, automation, and services. With OpenShift, your developers can make new containerized apps, host them, and deploy them in the cloud with the scalability, control, and orchestration that can turn a good idea into new business quickly and easily.

Best of all, OpenShift is supported and developed by the #1 leader in open source, Red Hat.


A look at how Kubernetes fits into your infrastructure

Kubernetes diagram

Kubernetes runs on top of an operating system (Red Hat Enterprise Linux Atomic Host, for example) and interacts with pods of containers running on the nodes. The Kubernetes master takes the commands from an administrator (or DevOps team) and relays those instructions to the subservient nodes. This handoff works with a multitude of services to automatically decide which node is best suited for the task. It then allocates resources and assigns the pods in that node to fulfill the requested work.

So, from an infrastructure point of view, there is little change to how you’ve been managing containers. Your control over those containers happens at a higher level, giving you better control without the need to micromanage each separate container or node. Some work is necessary, but it’s mostly a question of assigning a Kubernetes master, defining nodes, and defining pods.

What about docker?

The docker technology still does what it’s meant to do. When kubernetes schedules a pod to a node, the kubelet on that node will instruct docker to launch the specified containers. The kubelet then continuously collects the status of those containers from docker and aggregates that information in the master. Docker pulls containers onto that node and starts and stops those containers as normal. The difference is that an automated system asks docker to do those things instead of the admin doing so by hand on all nodes for all containers.

Source: https://www.redhat.com/en/topics/containers/what-is-kubernetes

3 Cool Linux Service Monitors

The Linux world abounds in monitoring apps of all kinds. We’re going to look at my three favorite service monitors: Apachetop, Monit, and Supervisor. They’re all small and fairly simple to use. apachetop is a simple real-time Apache monitor. Monit monitors and manages any service, and Supervisor is a nice tool for managing persistent scripts and commands without having to write init scripts for them.

Monit

Monit is my favorite, because provides the perfect blend of simplicity and functionality. To quote man monit:

monit is a utility for managing and monitoring processes, files, directories and filesystems on a Unix system. Monit conducts automatic maintenance and repair and can execute meaningful causal actions in error situations. E.g. Monit can start a process if it does not run, restart a process if it does not respond and stop a process if it uses too much resources. You may use Monit to monitor files, directories and filesystems for changes, such as timestamps changes, checksum changes or size changes.

Monit is a good choice when you’re managing just a few machines, and don’t want to hassle with the complexity of something like Nagios or Chef. It works best as a single-host monitor, but it can also monitor remote services, which is useful when local services depend on them, such as database or file servers. The coolest feature is you can monitor any service, and you will see why in the configuration examples.

Let’s start with its simplest usage. Uncomment these lines in /etc/monit/monitrc:

 set daemon 120
 set httpd port 2812 and
     use address localhost  
     allow localhost        
     allow admin:monit      

Start Monit, and then use its command-line status checker:

$ sudo monit
$ sudo monit status
The Monit daemon 5.16 uptime: 9m 

System 'studio.alrac.net'
  status                  Running
  monitoring status       Monitored
  load average            [0.17] [0.23] [0.14]
  cpu                     0.8%us 0.2%sy 0.5%wa
  memory usage            835.7 MB [5.3%]
  swap usage              0 B [0.0%]
  data collected          Mon, 04 Sep 2017 13:04:59

If you see the message “/etc/monit/monitrc:289: Include failed — Success ‘/etc/monit/conf.d/*'” that is a bug, and you can safely ignore it.

Monit has a built-in HTTP server. Open a Web browser to http://localhost:2812. The default login is admin, monit, which is configured in /etc/monit/monitrc. You should see something like Figure 1 (below).

Click on the system name to see more statistics, including memory, CPU, and uptime.

That is fun and easy, and so is adding more services to monitor, like this example for the Apache HTTP server on Ubuntu.

check process apache with pidfile /var/run/apache2/apache2.pid
    start program = "service apache2 start" with timeout 60 seconds
    stop program  = "service apache2 stop"
    if cpu > 80% for 5 cycles then restart
    if totalmem > 200.0 MB for 5 cycles then restart
    if children > 250 then restart
    if loadavg(5min) greater than 10 for 8 cycles then stop
    depends on apache2.conf, apache2
    group server    

Use the appropriate commands for your Linux distribution. Find your PID file with this command:

echo $(. /etc/apache2/envvars && echo $APACHE_PID_FILE)

The various distros package Apache differently. For example, on Centos 7 use systemctl start/stop httpd.

After saving your changes, run the syntax checker, and then reload:

$ sudo monit -t
Control file syntax OK
$ sudo monit reload
Reinitializing monit daemon

This example shows how to monitor key files and alert you to changes. The Apache binary should not change, except when you upgrade.

    check file apache2
    with path /usr/sbin/apache2
    if failed checksum then exec "/watch/dog"
       else if recovered then alert

This example configures email alerting by adding my mailserver:

set mailserver smtp.alrac.net

monitrc includes a default email template, which you can tweak however you like.

man monit is well-written and thorough, and tells you everything you need to know, including command-line operation, reserved keywords, and complete syntax description.

apachetop

apachetop is a simple live monitor for Apache servers. It reads your Apache logs and displays updates in realtime. I use it as a fast easy debugging tool. You can test different URLs and see the results immediately: files requested, hits, and response times.

$ apachetop
last hit: 20:56:39         atop runtime:  0 days, 00:01:00             20:56:56
All:           12 reqs (   0.5/sec)         22.4K (  883.2B/sec)    1913.7B/req
2xx:       6 (50.0%) 3xx:       4 (33.3%) 4xx:     2 (16.7%) 5xx:     0 ( 0.0%)
R ( 30s):      12 reqs (   0.4/sec)         22.4K (  765.5B/sec)    1913.7B/req
2xx:       6 (50.0%) 3xx:       4 (33.3%) 4xx:     2 (16.7%) 5xx:     0 ( 0.0%)

 REQS REQ/S    KB KB/S URL
    5  0.19  17.2  0.7*/
    5  0.19   4.2  0.2 /icons/ubuntu-logo.png
    2  0.08   1.0  0.0 /favicon.ico

You can specify a particular logfile with the -f option, or multiple logfiles like this: apachetop -f logfile1 -f logfile2. Another useful option is -l, which makes all URLs lowercase. If the same URL appears as both uppercase and lowercase it will be counted as two different URLs.

Supervisor

Supervisor is a slick tool for managing scripts and commands that don’t have init scripts. It saves you from having to write your own, and it’s much easier to use than systemd.

On Debian/Ubuntu, Supervisor starts automatically after installation. Verify with ps:

$ ps ax|grep supervisord
 7306 ?        Ss     0:00 /usr/bin/python 
   /usr/bin/supervisord -n -c /etc/supervisor/supervisord.conf

Let’s take our Python hello world script from last week to practice with. Set it up in /etc/supervisor/conf.d/helloworld.conf:

[program:helloworld.py]
command=/bin/helloworld.py
autostart=true
autorestart=true
stderr_logfile=/var/log/hello/err.log
stdout_logfile=/var/log/hello/hello.log

Now Supervisor needs to re-read the conf.d/ directory, and then apply the changes:

$ sudo supervisorctl reread
$ sudo supervisorctl update

Check your new logfiles to verify that it’s running:

$ sudo supervisorctl reread
helloworld.py: available
carla@studio:~$ sudo supervisorctl update
helloworld.py: added process group
carla@studio:~$ tail /var/log/hello/hello.log
Hello World!
Hello World!
Hello World!
Hello World!

See? Easy.

Visit Supervisor for complete and excellent documentation.