iTVmanager Training

Turning IPTV into

InterPersonalTV
1

2009 Minerva Networks, Inc. All Rights Reserved.

Session 5: System Administration &

Troubleshooting

Course Description
System

administration and troubleshooting.

Intended audience: Headend technicians, IT personnel,
anyone responsible in administrating and troubleshooting
iTVmanager and set-top-boxes.

2009 Minerva Networks, Inc. All Rights Reserved.

Course Objectives
Upon

completion of this course the student

should have a basic understanding of system
administrative tasks along with having the
skills and tools to recover from the most
common system issues and problems.

2009 Minerva Networks, Inc. All Rights Reserved.

Acronyms and Abbreviations

CSR Customer Service Representative

CA Conditional Access (video encryption)
PPV Pay Per View
EPG Electronic Program Guide
Tribune data EPG
PVR Personal Video Recorder
STB Set-Top Box
VOD Video On Demand
HD High Definition
2009 Minerva Networks, Inc. All Rights Reserved.

Main System Components of iTVmanager

Red-Hat

Linux 4.6
Oracle 10g Database
Oracle 10g Application Server
BackOffice Manager
EPG Digestion Process
Dispatcher (creates multicast streams)
Think Application (STB firmware)
6

2009 Minerva Networks, Inc. All Rights Reserved.

The BackOffice lets you manage the

follow components:

Multicast Streams
Customers
Devices
Channels & Packages
Regions and Lineups
Pay-Per-View
VOD Assets
Pricing on individual services (i.e., packages, CallerID, HD, email, web)
Client Messaging
Caller ID
Emergency Alert System (EAS)
Billing

2009 Minerva Networks, Inc. All Rights Reserved.

Topic: Adding BackOffice Users

1.
2.

Go to System Setup and then to the Admin User tab

Select the Add button which will bring up the Add New Admin User page.

2009 Minerva Networks, Inc. All Rights Reserved.

Adding BackOffice users continued:

Fill out the appropriate user information. The fields in red are required and all other are
optional.
Under Access Modules select the appropriate access rights for the new user. For
example, CSRs might only be given Customers, Devices, & Billing. Marketing people
may only need Channel Mgmt, Service & Pricing.
Please be sure to enforce strict access rights to the BackOffice except for a chosen few
individuals

2009 Minerva Networks, Inc. All Rights Reserved.

Topic: Customer Service Messaging

10

Customers can be sent

instant or scheduled
messages that appear in a
popup window on the TV
screen and in the Vault.

Messages are sent via the

Message Center module

Messages can be sent to

all customers at one time or
to selected customers

If no date and time is

entered the message will
occur instantly.
2009 Minerva Networks, Inc. All Rights Reserved.

Topic: Adding Multicast Addresses

11

The IP Headend / Setup module is a multifunctional admin page where certain system
wide defaults are set such as the maximum number of channels that can be stored on the
system, the maximum amount of EPG data that can be stored in the database and on
STBs, and where system wide STB reboots are initiated
This page is where new multicast IPs and ports are defined

2009 Minerva Networks, Inc. All Rights Reserved.

Parameter Definition- IPHeadend page

12

Channel Information Where the maximum number of channels and

multicast IPs are defined
Scheduled Load Time The time the EPG data is ingested into the database
Scheduled Blocks Time increments used to display EPG data in the EPG
module in the Channel Manager
Scheduled Days forward / back Amount of EPG data stored (in days) on
the system database
Cache Duration Amount of EPG stored on the set-top-boxes
Set Top Box Where to schedule or make unscheduled system wide STB
reboots
Multicast Address Where multicast addresses are defined
Multicast Port Where multicast ports are defined
Messaging Server The IP of the system where the messaging process is
running
Bandwidth reservation Not available yet

2009 Minerva Networks, Inc. All Rights Reserved.

Three steps to make a multicast IP available

on the system:

13

1.

The address is created in the IPHeadend page in the

Multicast Address field

2.

In the Multicast Address field only the first three octets of the
IP is given follow by the number of IPs to be created. If you
want one IP enter the entire IP follow by a 1 in the number of
addresses box.

3.

The address or addresses then must me add to a record in

the Multicast Addr module (next slide)

2009 Minerva Networks, Inc. All Rights Reserved.

assigning multicast addresses to a record:

1.
2.
3.

14

In the Multicast Addr page, select the new address(s) in the Available Multicast
IP box
Select the Assign and then Submit button
The multicast address will now be available in the channel manager

2009 Minerva Networks, Inc. All Rights Reserved.

Topic: Setting up the Primary Dispatcher

The primary dispatcher

creates a set of steams
needed for STB boot-up:

15

Runtime stream (EPG &

VOD updates)
Service messaging (EAS, &
all system wide
announcements)
STB Boot Stream (carries
most of the boot information)

Multiple boot streams can be

created to display various
STB background themes (i.e.,
Swirl or Default)
2009 Minerva Networks, Inc. All Rights Reserved.

Topic: Troubleshooting and iTVmanager

processes

16

Useful Software Tools:

VLC media player (http://www.videolan.org/vlc/)
Putty a telnet and SSH client (http://www.putty.org/)
Ethereal or Wireshark (http://www.wireshark.org/)
Admin Users:
The three main users to manage the system are: root, itvmgr, &
oracle
Important System Admin Scripts:
Itvmgrctl - for stopping and starting iTVmanager process
Stop10g.sh & start10g.sh - scripts for bouncing the data-services
process or Oracle10g
TCPDUMP Utility that prints out packets and other activities on a
server network interface.
2009 Minerva Networks, Inc. All Rights Reserved.

iTVmanager Processes & Usage:

The itvmgrctl script is used to stop and start all

iTVmanager processes running on the Application server.
In order to use this script you must be logged in as the
itvmgr user. Below is an example of how to get the
running status of each process. su itvmgr is the
command to change users. SU stands for switch user.

17

2009 Minerva Networks, Inc. All Rights Reserved.

Here is an example of how to stop and start all processes. Follow

the commands in the graphic below and will be a success:

18

Example of stop and restarting all iTVmanager process:

2009 Minerva Networks, Inc. All Rights Reserved.

iTVmanager Processes

19

Dispatcher : Creates the multicast boot streams. Sends data to the STB from
the BackOffice (i.e. boot information, EPG data, system announcements, etc.)
VODDS (VOD Delivery System): Manages the over all VOD ingestion process
between the catcher and Minerva.
VSM (Video Server Manager): Mainly manages the content that is placed on the
video server.
Mesagpoller: Handles all the messaging for example EAS and customer
messaging.
CallerInfo: Handles all the processing for CallerID
DataServices (Oracle10g): This process is not controlled by the itvmgrctl
script. It handles interactions between the database and other iTVmanager
components (i.e. communication between STBs and the Oracle database). Also
some GUI pages are served to the BackOffice via this process (i.e., Devices,
Customers, Channel Manager, and the System Configuration page).

2009 Minerva Networks, Inc. All Rights Reserved.

How to bounce iTVmanager Processes and Oracle10g:

First step in the recovery of system wide STB boot failure

1.
2.
3.
4.
5.
6.

20

su itvmgr (Login as itvmgr user)

itvmgrctl stop all (Stop all iTVmanager
processes)
cd /itvmgr/bin
./stop10g (Stop Oracle 10g application server)
./start10g (Start Oracle 10g application server)
itvmgrctl start all (Restart iTVmanager
processes)
2009 Minerva Networks, Inc. All Rights Reserved.

TCPdump and VLC or Wireshark

A useful combination to prove or not to prove if all STB boot

streams are being sent across the network is to sniff on the
access network and at the same time run TCPDUMP on the
iTVmanager server

Minerva Server
eth0
Laptop running VLC
connected to the STB network

21

Console or SSH connection

TCPDUMP

2009 Minerva Networks, Inc. All Rights Reserved.

TCPDUMP Usage

Useful tool to determine if the STB boot streams are streaming out of
the iTVmanager network interface. If not set-top-boxes will not boot.
The follow commands are used on the server running the
dataservices process.

Output is only the boot streams which are on port 4545

# tcpdump -i eth0 port 4545

Sniffing for a specific bootstream

# tcpdump host 239.255.5.5 -i eth0 vvv

Displays all TCPIP activity on eth0

# tcpdump n i eth0

22

2009 Minerva Networks, Inc. All Rights Reserved.

iTVmanager System Logs

All Minerva system log files are located in the /itvmgr/log directory on the
Application server
1. Adminlog (BackOffice GUI)
2. Adminusagelog (system user usage information)
3. Dataserviceslog (Oracle App10g)
4. Dispatcherlog (multicast stream information)
5. Mesgpollerlog (EAS & customer messaging)
6. Voddslog (video on demand)
7. Vsmlog (video System Manger)
To display real-time information in any log use the following command:
# tail f /var/log/logfilename

23

2009 Minerva Networks, Inc. All Rights Reserved.

Some Important Maintenance Checks:

Check server date and time often to ensure EPG accuracy

# date (use to display date and time)
# date s 12:00:00 (use to change time)
- Setup the NTP process in /etc/ntpd.conf

Check hard-drive space and verify no file partitions are close to filling up.
A full partition could crash a server or important processes
# df h

Check EPG download report and verify two-weeks of data is present.

:See the Billing and Cron handout for instructions on how to manually download EPG.

24

Make sure the nightly database is backing up seven days

2009 Minerva Networks, Inc. All Rights Reserved.

How to adjust EPG stored on set-top-boxes:

Different brands of set-top-boxes can handle different amounts of EPG

To set the amount of EPG data that displays in the guide, do this:
1. Log into the BackOffice Console.
2. Go to System Setup > Configuration. In the Configuration
Parameters list, scroll to the thinkstuff_STB module (i.e.,
thinkstuff_amino, or thinkstuff_adb, etc) and edit this file.
3. Set the max_epg_data parameter to the desired amount of
data. The adjustment is in hours.
4. On the IP headend > Setup page set the Cache Duration
parameter to the same value or higher. This setting is the
maximum amount of EPG that any brand of STB can store.

25

2009 Minerva Networks, Inc. All Rights Reserved.

Things to be cautious of:

26

Never change the host IP address(eth0) on the middleware

servers without using the Minerva ChangeIP script. Please
consult Minerva before attempting to change host IPs.
Do not install OS periodic updates from the Red-Hat website
without consulting Minerva first. Installing updates could add
OS and/or kernel components that are not compatible with
Minerva software
Not logging out of BackOffice GUI can lock up certain records
which results in others not being able to modify them
Bouncing Minerva processes without warning other people in
your organization can cause BackOffice users to loose their
work or cause set-top-box installation issues
2009 Minerva Networks, Inc. All Rights Reserved.

References
Minerva

troubleshooting guide
Maintenance checklist document
Boot step and error code document

27

2009 Minerva Networks, Inc. All Rights Reserved.

Thank You!

28

2009 Minerva Networks, Inc. All Rights Reserved.