אינדקס
PGI 1.5, Baragon Reply to this post Create Quote Reply To This Post Edit / Delete this post Report this post to a Moderator Go to the top of the page
Changes in PGI 1.5
============================================== ===================
- Fix: EPG-column in the channel list did not appear immediately
- Fix: When a recording ends on a channel while you are in the time shift, the picture remains black
- Fix: The sleep timer could not be disabled once it has been turned on
- Update: Busybox and transponder database updated USB version
**** READ THIS README BEFORE FLASHING THE NEW IMAGE *****
THIS SOFTWARE AND README IS PROVIDED "AS IS" AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. UNDER NO CIRCUMSTANCES WILL THE RECCONV SOFTWARE AUTHOR BE LIABLE FOR ANY INDIRECT, INCIDENTAL, CONSEQUENTIAL, SPECIAL OR EXEMPLARY DAMAGES, DATA LOSS OR HARDWARE MALFUNCTION, ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE UNDER NEGLIGENCE OR EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. BY INSTALLING AND USING THE SOFTWARE YOU ACCEPT ALL THE TERMS AND CONDITIONS OF THIS AGREEMENT.
README for converting old format movies to new format movies.
Whenever the readme speaks of old (format) recordings, this means: Recordings made with PGI 0.7.2a or before (or any DGS R12413 and before)
Whenever the readme speaks of new (format) recordings, this means: Recordings made with PGI 0.8 or later (or any DGS R13078 and later untill they change the format again)
This readme describes the converter that gives the option to convert old recordings to the new format.
Recconv 1.0 Final
=================
This will be the last version of the recording converter, as after a couple of months with the newer images everyone should have converted their recordings, either
with 0.8, 0.8.1 or with 1.0. The s/w changes everytime and it is a lot of work to keep a one time tool up to date (this is for example the reason the converter failed in 0.9).
Except the improvements below, nothing changed in the way to use it, so see below.
Improvements
============
- Now handles special characters correctly
- The restarting of the conversion has been improved, the broken off conversions will be deleted prior to converting them again
- recconv will now be daemonized after starting and answering the questions. Wait till recconv says: You can now close this telnet session. You can turn off you pc as well after doing this
- Converted recordings will be appended with _c in the file name so you can recognize them easily. Recordings that were converted succesfully with the older versions of recconv will not be converted again.
Extra remarks:
==============
- After booting PGI 1.0 wait at least 5 minutes before starting the conversion
- READ THE REST OF THE README!!!
Prerequisites:
--------------
Before Flashing the new image:
If you use post descrambling, turn it off and make sure all old recordings are descrambled before flashing the image. Scrambled recordings cannot be converted.
If you have timeshift on, turn it off, just to be safe.
If you have any timers set, delete those timers as they might boot up the box while converting the recording.
As converting is at realtime speed, you might want to consider deleting unneeded recordings prior to flashing.
You can now safely flash the image.
After flashing the image, do not open the mediafile list prior to running the converter, or before manually deleting the /media/filelist.meta file. If you open the mediafile list before doing this, the box will panic and might corrupt your db.dat
Features/remarks
----------------
The converter converts the old recordings at about the same speed as that a post descrambling, so the best time to run it will be at times when noone is using the box.
All previous bookmarks, skips and repeats are lost during conversion. This is due to the new file formats and it isn't possible to easily convert the bookmarks etc.
The converter will show its progress on the VFD on at least the 9000 and CubeRevo models, the VFD option is not tested on other boxes, so it might just as well show garbage, but at least it will change every second.
The VFD will show the following:
| 1/10 20%
The first character (in this case | )will make a step every 1 seconds this way you can see that the application is still running. Next you will see the recording being converted as in "1 of 10". As the last info you get an indication of how far the conversion is for this specific recording. This is just an indication, not exact.
To prevent overheating of the box, the fan will be turned on during conversion, keep this in mind when choosing the time at which you want to convert (for example if the box is in the bedroom).
The converter can be restarted after a reboot and will skip already converted recordings, please see conversion step 9
The box MUST be rebooted after conversion, even after breaking off a conversions.
During the converion, a log file is generated: /media/converter.log This log shows what is happening and if any failures occured if a failure occurs, the old format recording won't be deleted.
Requirements:
-------------
- a network connection and a pc with putty or another telnet client. Do NOT use web interface to execute Telnet commands!
- Diskspace should be at least 1.5 times the size of your longest recording. If you plan to keep the old format recordings, make sure you have at least 52% of free diskspace (see Conversion step 4).
- If you have enough diskspace, keep the old format recordings and delete them manually after checking the converted recording.
Conversion steps:
-----------------
Steps to convert the recordings (all after flashing the new image):
1) the box needs to be in stand by mode (WARM OFF), to get it there, press the power button on the remote briefly so that you get the powerdown menu (you might need to enable the menu if you have disabled it in the PGI settings). Choose the option Standby (the bottom one).
*** If the box isn't in standby mode, the conversion will fail!
If you have forgotten, the converter will automatically switch the box into standby mode before step 8.
2) The box will go into standby mode now.
3) telnet to your box
4) Check your free diskspace by typing:
'df'
and look for the line containing /dev/sda2
You need at least 1.5 times the free disk space as your longest recording. A value of 20 GB would suffice in most cases. If you plan to keep the old format recordings, you need at least 52% free disk space. The converter application doesn't check the available diskspace, so your are on yourself here, running out of diskspace during conversion will cause problems that looses the specific recording it is converting.
5) start the converter by typing:
'recconv'
(without the quotes and press enter)
6) The application will now ask you whether or not to keep the old format recordings, type either Y or N. Best would be to keep them after converting them, unless you plan to switch back to a previous (0.7.2a and before) version of PGI. Besides that, if conversion fails and isn't handled correctly, you can always rerun it
7) If you selected N at step 6, the converter will ask you to type the word 'delete' (without quotes) to make sure that you really want to delete them. Anything else then 'delete' will exit the converter.
The conversion will now start.
The box will automatically switch to warm off, then after 5 seconds the conversion will start (in case you wonder if anything happens, there is a 5 second delay before starting the conversion).
During conversion you can see the progress on the VFD.
9) If you want to break off the conversion (for example in the morning and the conversion isn't finished yet). Make a second telnet connection to your box and run 'stopconvert.sh' without the quotes.
The box will reboot now. The conversion can be restarted by starting at step 1 again any previous converted recordings will be skipped.
10) When the conversion is finished, the VFD will show Finished
11) When finished, the box will reboot itself automatically.
The conversion is now finished, after the reboot, your recordings will show up in the mediafile list (might be that they don't show up at opening the list the first time, but they will when closing the list and reopening it after a few minutes, populating the list is done in the background and might not have finished yet when you open it the first time).
If you have any questions (even before converting), please post them at denktenk.com
Manually deleting the old format recordings:
--------------------------------------------
If you decided to keep the old recordings and want to remove them after checking the converted recordings, you can do the following
telnet to your box
File by file:
WARNING: take a good look at the file name, old recording directory starts with ~ this is no typo!!! forgetting it will delete the new recording without a possibility to recover!!!!
cd /media
rm -r ~recordingname
rm recordingname.ifo
rm recordingname.dgsifo
rm recordingname.extinfo
You can also use the above to create diskspace
If you feel daring and want to to them all at the same time:
WARNING: take a good look at the commands below, the ~ is not a typo, leaving this one out erases you entire harddisk!!!!
cd /media
rm -r ~*
rm *.ifo
rm *.dgsifo
rm *.extinfo
Reconverting an already converted recording:
--------------------------------------------
If you find a recording that hasn't converted correctly (for example filesize is 0) you can do the following:
cd /media
rm -r name_of_recording
WARNING: the name_of_recording is the new one, don't delete ~name_of_recording as that is the old format version!!! The difference is the ~ this is not a typo.
Hard Disks, Flash Media, Formatting and Swap Space
===============================================================================
0.0 Storage device initialisation in PGI differs from other software images. The differences DO NOT AFFECT the devices in a standard configuration which is the following: ONE storage device (external USB or internal SATA) which has already been formatted by one of the DGS based images and was used for making the recordings before. If your configuration differs from the described or you are having troubles connecting some new device or you have custom partitions layout (for multiboot, skins or EPG storage only on USB sticks, etc.), then you should take your time and read the notes below.
0.1 PGI WILL NEVER FORMAT YOUR DISKS OR FLASH MEDIA BY ITSELF. Which means that if you want to connect new storage device like USB Flash or new SATA disk that has never been used before by the receiver, they will NOT be formatted and recognised automatically. This decision was deliberate and the main reason behind it was that while automated formatting may make life easier for some people, it may destroy valuable data for others who wish to use their connected devices for more than just pure PVR functionality. Since the software cannot magically guess what you intend to do with your storage device, you will have to choose device formatting option by hand when desired. Luckily, with PGI that procedure is quite straightforward.
0.2 In contrast with other images, the OSD menu called "HDD Settings" ALWAYS stays active. Therefore, if you want to format newly installed (or even an existing) device, you can do it at any time from that menu. Please pay attention to the message that is displayed once you choose the formatting option. The message will display the actual information about the device that will be formatted. If that device is not what you expect it to be, read the next paragraph.
0.3 What exactly will be formatted? This question may seem strange at first but only to those who have only one storage device connected. If you have more than one device connected (like SATA and USB together), then you have to be sure you are formatting the right disk. As a general rule, the disk which contains your /MEDIA partition will be formatted. Note that the whole disk with ALL(!) of its partitions be erased, not just the /media partition, so if you have non-standard disk layout with multiple partitions (used for multiboot, etc) you should never format your disk using receiver's OSD menu. In detail, the following occurs during format: first, ALL PARTITIONS will be deleted from the device that contains your /media partition and 2 new partitions will be created after that. The first partition (for example, /dev/sda1) will always be 256MB in size and will be used as a swap space (equivalent of the Windows' page file). The second partition (for example, /dev/sda2) will take the rest of the available disk space and will be used for the /media partition (i.e., PVR functions such as timeshift and channel recording). By default the ext3 file system will be used to format the /media partition. If you prefer ext2 instead, you can set the HDD_FS parameter in pgi.conf accordingly (see HDD_FS description below as well as the next paragraph)
0.4 Which file system is better: ext2 or ext3? The answer will be "it depends". ext2 performs slightly faster than ext3 which may become a crucial point for some devices with slower write speed. At the same time, ext3 handles errors much better since it is a journaling file system (and that makes it slightly slower). You have to try it for yourself. If your setup can handle ext3 it is better to stick with it rather than with ext2. If the write speed is not enough you will get "freezes" and "hiccupping" during timeshift and recording, especially with the high bitrate HD channels. For most SATA devices ext3 is fine.
0.5 Can I use another file system (not ext2 and not ext3) on my storage device? The short answer is no, not for recording. The longer answer is that you can only use ext2 or ext3 for recording but PGI images also understand FAT and NTFS file systems in the "read-only" mode, i.e. you can play media from USB flash formatted as FAT or connect your NTFS formatted HDD with the media files to view them but you will not be able to use devices with such format for channel recording. Why ext2 or ext3? Because they are native Linux file systems and are optimised for speed and error handling. For this very reason you cannot read the disks that you use for recording with Windows, if you decide to connect them to a PC later. There is some third-party Windows software available (some free and some not) to be able to read Linux file system like ext2 or ext3 under Windows. Why IPBox does not use more advanced Linux file systems such as ext4 or xfs? Because they require too much resources than what's available on IPBox and will be very slow or will not work at all.
0.6 If you have more than one storage device connected or you wish to use some non-standard disk layout with multiple partitions, you can select which partition should be used for /media via the MEDIA_DEVICE setting in pgi.conf. To make proper use of MEDIA_DEVICE setting it is mportant to determine the names of your storage devices. If you have only ONE device connected (does not matter if it's a SATA or a USB device) then that device will be named /dev/sda (and the partitions on it will be named /dev/sda1, /dev/sda2, etc.) If you have more than one storage device connected, SATA device will always have higher priority in terms of naming and usually will be called /dev/sda while any additional external USB disks will be then named /dev/sdb, /dev/sdc and so on. Finally, if you have several external USB devices connected at the same time, the order of their naming could change unpredictably from one boot to another (depending on how fast they reset during boot). In such (rare) situations you can use device's UUID (unique identifier) to specify the required partition for /media. To obtain UUID for all your existing EXT2/EXT3/FAT partitions you can execute "blkid" command in the Telnet session (also explained below in the MEDIA_DEVICE description).
0.7 PGI 1.x software uses most of the receiver's free RAM, especially when working with lots of EPG data. Therefore, it is highly recommended to have swap space enabled for the image to work properly. PGI will automatically detect all of the swap partitions that you may have and will make use of them without any additional configuration. For example, if you have internal HDD or external USB storage attached and formatted using receiver's OSD menu "Format HDD", then the swap space will be allocated as the 1st partition on the disk (and 2nd partition will be used as /media). If you have custom disk layout, you can allocate ANY partition for your swap space (you need to run mkswap command on that partition to mark it as swap) and it will be automatically detected and mounted during boot without changing any settings, no matter which partition it is going to be. Additionally, PGI image allows to use any ordinary file as a swap space. This means that if you want to have custom disk layout, you do not necessarily need to make a dedicated swap partition but can use an empty file on any other partition instead. If you decide to go ahead with a file as your swap space, you need to specify its full path in the SWAP_FILE parameter in pgi.conf (see its description below). WARNING: NEVER EVER use receiver's flash memory for swap, i.e. never assign files in /var directory for such purpose! It is also not recommended to use /media partition to hold your swap file because HDD health check that runs once in a while during boot may fail due to the fact that /media needs to be unmounted before checking (which means swap file cannot be used, which means you may run out of memory if the /media partition is too large in size).
0.8 If your receiver does not have ANY storage device attached internally or externally (usually the case with Cuberevo Mini/250/200 models), it is highly recommended to connect any USB stick (the faster speed, the better) and use it as a swap device. As an added bonus you will have possibility to store things like persistent EPG data, skins and picons on the same device. The USB stick does not need to be huge size if you are not planning to use it for channel recording; even 1GB stick would be more than enough to hold your swap space, EPG data, picons and skins. There are two possibilites with regards to how format such USB stick. One is to format it using receiver's "HDD Setup" menu, just like you would do it with a hard disk for PVR functions. In this case you will not need to change any default parameters in pgi.conf as both your swap and your /media partitions will be now on your USB stick (just don't forget to disable the Timeshift in the PVR options as soon as you have formatted the stick!) Alternatively, you can use ordinary FAT32 formatted USB stick with 1 partition on it, without reformatting it. In this case you need to change the following pgi.conf settings: SWAP_FILE, EPG_DATA_DIR and, if required, PICONS, SKIN and BACKUP_DIR to point to "/mnt/usb/sda1/...". The preferred way is the first one, because Linux native EXT3 file system will perform better than FAT32 in most scenarios.
0.9 To see the list of all partitions and/or files which mounted as swap you can use the following Telnet command: "cat /proc/swaps", to check the swap space usage you can use the following Telnet command: "free" (commands need to be typed it without quote marks). The "free" command shows more than just swap usage information, but you are only interested in the last line of the output that starts with "Swap:". It should contain three numbers. The first one is the total amount of swap memory (should be greater than 0 if swap is enabled) the second number is the amount of swap memory used at the moment (ideally it should be as low as possible), the last number is the amount of free swap space.
My Disk IS Not Being Recognised!
===============================================================================
1.0 You can always see some information about why your /media partition is not being recognised. The information is available in the "HDD Info" OSD menu and you should always start any troubleshooting from that screen. Most of the times it will be immediately apparent what went wrong just from the status message in that screen. If that information is not enough then you can enable debugging in pgi.conf (DEBUG="yes") and check the /var/log/hdd.log file after the reboot for most detailed information. Below you will find the description of most common errors and messages that you can see in the "HDD Information" menu.
1.1 Very often people think that their HDD has not been recognised, while actually it is simply being checked for errors during startup. The error checking may take a long time if you have many files (or many errors) on your disk and during this time the HDD will be "unavailable". You can see the disk checking progress in the "HDD Information" menu. The procedure of disk checking has 5 stages (and EACH stage goes from 0 to 100%). The disk checking can be enabled and disabled via the "HDD Settings" menu. It is a very good idea to check your disk once in a while to avoid data loss or corruption which could come from an unexpected reboot or panic. Generally speaking, the error checking is performed MUCH faster on ext3 file systems.
1.2 If you see the error that says "/dev/sda2 cannot be detected and no NFS is set" it means that the receiver cannot actually detect ANY storage devices connected to it (and you are not using NFS recording). There are 2 most common reasons for this:
1.2.1 If you get this message after you have installed new SATA disk, then chances are that the actual hardware is simply incompatible. Some people had success when they switched the HDDs from SATA 2 into SATA 1 mode by using a special jumper on their HDD.
1.2.2 If you get this message while using a USB HDD, the reason for the failure could be that it takes longer for the HDD to "spin up" after the cold start and the receiver checks for this device "too early". To overcome this problem you can force device detection (the device timeout will be longer) by setting HDD_USB_DETECT parameter in pgi.conf as follows: HDD_USB_DETECT="force" (see detailed description of this parameter below). If the device cannot be initialised even with such option, then you will see an error "/dev/sda detection timeout" in the "HDD Information" menu. In such case the only thing left to do would be to try another USB HDD model or provide external power to your USB disk if you have not done so.
1.3 If the error message says "/dev/sda2 does not exist", it means that your HDD *is* being detected, but is not partitioned the way IPBox expects it, i.e., second partition, is normally expected to be used for PVR tasks and it does not exist. There are two possible situations that may cause the error:
1.3.1 You device has never been used before for PVR functionality so it does not have correct partition layout. If you DO want to use this device for recording or timehift, you should simply reformat it using standard OSD menu option (see the beginning of this document).
1.3.2 You have a single USB device connected but do NOT wish to use it for PVR functions. Receiver would normally assume that the first detected device (sda) should be used for recording and will try to detect 2nd partition which is normally used for such purpose. Since in "receiver's opinion" your device is not partitioned the "right way" for PVR, you will get an error. In such case it is recommended to set the HDD_USB_DETECT parameter in pgi.conf as follows: HDD_USB_DETECT="no". This will prevent your receiver from trying to use your device for PVR functions.
1.4 If the error message says "/dev/sda2 has FAT file system" (or "NTFS file system" or "unknown system") that means that the device *is* detected, *has* the second partition, but the partition is not formatted using ext2 or ext3 file system. In such a case you need to format the device (see the beginning of this document).
1.5 If the error message says "/dev/sda2 cannot be mounted" that means that the device has correct format but due to some inconsistency or data corruption cannot be mounted. In such case it is best to attempt to run disk error checking by selecting this option from the "HDD Settings" menu and rebooting (you can watch the progress as described in 1.1 above). If that does not help, then reformat may be the only option.
1.6 The error "Cannot create tv/radio/multi dir" means that the device has been mounted, but no data can be written to it. Most often this happens with incorrectly set up NFS share used for recording when the /media device is mounted in a read only mode.
1.7 If the error message has something like "fdisk failed" or "mkfs failed" or "mkswap failed", that means that the HDD format was attempted but failed. You can try formatting one more time and if that still fails then it is worth checking the full log file as described in 1.0 above.
1.8 If the error is "NFS attempted but failed", then the connection to NFS recording share was attempted but failed. It is best to try to manually mount the same share from the Telnet session to see why it is not working. You should also check the correctness of your NFS path in MEDIA_DEVICE parameter (note that the colon sign is used twice for NFS paths).
Storage Device Related Parameters for pgi.conf
===============================================================================
In addition to the parameters listed in the REAMDE_en.txt file, there are some more HDD related options that can be set if required. It is best to only use them when you fully understand what you are about to do. As always, if the changes to these parameters cause non-booting receiver or data loss, it is your own problem.
HDD_AWAKE
By default, the shutdown sequence will try to spin down HDD shortly before shutting down power. If this feature is problematic for your configuration (some externally powered USB disks do not wake up until their power is reset), you can disable it by setting this parameter to "yes" in which case no command to spin down HDD will be sent.
Possible values: "yes", "no"
Default value: "no"
Example: HDD_AWAKE="yes"
HDD_FS
The name of the file system to use for /media partition formatting. Setting this option WILL NOT re-format the existing system and will not magically convert from one file system to another. You will have to manually invoke format from the OSD menu to change the file system (and lose all your data with it)! If this option is not set, "ext3" will be used the next time you format the storage device.
Possible values: "ext2", "ext3"
Default value: "ext3"
Example: HDD_FS="ext2"
HDD_SLEEP
The time in MINUTES after which the HDD should start sleeping, if inactive for that amount of time. "Inactive" means: there is no channel recording going on, timeshift is off and no EPG data, skins or picons is being accessed at the moment.
Possible values: 0 (no sleeping), 1 .. 20 and then 30, 60, 90, 120... etc.
Default value: "10"
Example: HDD_SLEEP="3"
HDD_USB_DETECT
Helps to solve problems with USB HDD detection in certain configurations. Normally receiver tries to autodetect external USB storage device automatically but sometimes autodetection does not work as expected. If your USB HDD does not get recognised at all during boot sequence you can try to set this parameter to "force" to enable alternative detection procedure. Also, if you have only 1 storage device permanently connected to your receiver but do NOT want to use it for PVR functions set this parameter to "no" to avoid false alarms related to "incorrect disk partitioning". However, for most configurations the "auto" setting works just fine. If you have problems recognising SATA drives, this is NOT the parameter that is going help and you should leave it at "auto" (see 1.2.1 above).
Possible values: "auto", "force", "no"
Default value: "auto"
Example (for problematic USB HDD that does not get detected nicely): HDD_USB_DETECT="force"
Example (if the *only* device is USB storage device and is *not* being used for PVR functions): HDD_USB_DETECT="no"
SWAP_FILE
Allows to specify a file that should be used as a swap space. The file MUST already exist (will not be created automatically) and it MUST be marked as a swap space. Both actions (creating an empty file of required size and marking it as swap) can be done using Telnet commands. For instance, the following commands will create 128 MB empty file in /mnt/usb/sda3 called pgi.swap and then it will be marked as swap space:
mkfile 128m /mnt/usb/sda3/pgi.swap
mkswap /mnt/usb/sda3/pgi.swap
Afterwards this file name can be specified as SWAP_FILE and it will be used IN ADDITION to all autodetected swap partitions on all connected disks. See 0.7-0.9 above for more information on swap space.
Default value: "none" (i.e. no special file used for swap, only the autotedected real swap partitions will be used)
Example (swap file on one of the partitions of an internal SATA disk): SWAP_FILE="/mnt/usb/sda3/pgi.swap"
Example (swap file on an external USB stick): SWAP_FILE="/mnt/usb/sdb1/myswap.tmp"
MEDIA_DEVICE
Allows to specify partition on SATA/USB device or NFS share which will be used as the /media folder (i.e. PVR functions such as channel recording and timeshift).
There are TWO possible ways to specify the SATA/USB device: using /dev/sdxX syntax and using device's UUID (see examples below). To see device UUID you can use 'blkid' command in the Telnet session.
There are TWO possible ways to specify an NFS share: when your NFS server uses UDP the path should start with nfs:// and when the server uses TCP the path should start with nfs-tcp:// (see the examples below).
A tip: you do NOT need to set NFSD="yes" in pgi.cong for NFS recording to work!
Default value: "/dev/sda2"
Example (SATA or USB storage device): HDD_MEDIA="/dev/sda3"
Example (SATA/USB storage device UUID): HDD_MEDIA="b7dfba7e-0e6b-4066-8906-d344e5a43912"
Example (NFS share using UDP protocol): HDD_MEDIA="nfs://192.168.1.2:/myshare"
Example (NFS share using TCP protocol): HDD_MEDIA="nfs-tcp://192.168.1.3:/myshare"
WARNING! If you change the default MEDIA_DEVICE setting and then format your HDD via the standard "HDD Setup -> HDD Format" menu, you have to remember 2 things:
1) Not only /media partition will be formatted, but the complete disk which contains media partition will be erased with ALL of its partitions!
2) The layout of such formatted disk will always be standard: 1st partition will be used for swap space, 2nd for media.
If you require different layout, never format your disk using the "HDD Setup -> HDD Format" menu and do it manually instead.
Enjoy your recordings,
PGI Team
Changes in PGI 1.5
============================================== ===================
- Fix: EPG-column in the channel list did not appear immediately
- Fix: When a recording ends on a channel while you are in the time shift, the picture remains black
- Fix: The sleep timer could not be disabled once it has been turned on
- Update: Busybox and transponder database updated USB version
**** READ THIS README BEFORE FLASHING THE NEW IMAGE *****
THIS SOFTWARE AND README IS PROVIDED "AS IS" AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. UNDER NO CIRCUMSTANCES WILL THE RECCONV SOFTWARE AUTHOR BE LIABLE FOR ANY INDIRECT, INCIDENTAL, CONSEQUENTIAL, SPECIAL OR EXEMPLARY DAMAGES, DATA LOSS OR HARDWARE MALFUNCTION, ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE UNDER NEGLIGENCE OR EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. BY INSTALLING AND USING THE SOFTWARE YOU ACCEPT ALL THE TERMS AND CONDITIONS OF THIS AGREEMENT.
README for converting old format movies to new format movies.
Whenever the readme speaks of old (format) recordings, this means: Recordings made with PGI 0.7.2a or before (or any DGS R12413 and before)
Whenever the readme speaks of new (format) recordings, this means: Recordings made with PGI 0.8 or later (or any DGS R13078 and later untill they change the format again)
This readme describes the converter that gives the option to convert old recordings to the new format.
Recconv 1.0 Final
=================
This will be the last version of the recording converter, as after a couple of months with the newer images everyone should have converted their recordings, either
with 0.8, 0.8.1 or with 1.0. The s/w changes everytime and it is a lot of work to keep a one time tool up to date (this is for example the reason the converter failed in 0.9).
Except the improvements below, nothing changed in the way to use it, so see below.
Improvements
============
- Now handles special characters correctly
- The restarting of the conversion has been improved, the broken off conversions will be deleted prior to converting them again
- recconv will now be daemonized after starting and answering the questions. Wait till recconv says: You can now close this telnet session. You can turn off you pc as well after doing this
- Converted recordings will be appended with _c in the file name so you can recognize them easily. Recordings that were converted succesfully with the older versions of recconv will not be converted again.
Extra remarks:
==============
- After booting PGI 1.0 wait at least 5 minutes before starting the conversion
- READ THE REST OF THE README!!!
Prerequisites:
--------------
Before Flashing the new image:
If you use post descrambling, turn it off and make sure all old recordings are descrambled before flashing the image. Scrambled recordings cannot be converted.
If you have timeshift on, turn it off, just to be safe.
If you have any timers set, delete those timers as they might boot up the box while converting the recording.
As converting is at realtime speed, you might want to consider deleting unneeded recordings prior to flashing.
You can now safely flash the image.
After flashing the image, do not open the mediafile list prior to running the converter, or before manually deleting the /media/filelist.meta file. If you open the mediafile list before doing this, the box will panic and might corrupt your db.dat
Features/remarks
----------------
The converter converts the old recordings at about the same speed as that a post descrambling, so the best time to run it will be at times when noone is using the box.
All previous bookmarks, skips and repeats are lost during conversion. This is due to the new file formats and it isn't possible to easily convert the bookmarks etc.
The converter will show its progress on the VFD on at least the 9000 and CubeRevo models, the VFD option is not tested on other boxes, so it might just as well show garbage, but at least it will change every second.
The VFD will show the following:
| 1/10 20%
The first character (in this case | )will make a step every 1 seconds this way you can see that the application is still running. Next you will see the recording being converted as in "1 of 10". As the last info you get an indication of how far the conversion is for this specific recording. This is just an indication, not exact.
To prevent overheating of the box, the fan will be turned on during conversion, keep this in mind when choosing the time at which you want to convert (for example if the box is in the bedroom).
The converter can be restarted after a reboot and will skip already converted recordings, please see conversion step 9
The box MUST be rebooted after conversion, even after breaking off a conversions.
During the converion, a log file is generated: /media/converter.log This log shows what is happening and if any failures occured if a failure occurs, the old format recording won't be deleted.
Requirements:
-------------
- a network connection and a pc with putty or another telnet client. Do NOT use web interface to execute Telnet commands!
- Diskspace should be at least 1.5 times the size of your longest recording. If you plan to keep the old format recordings, make sure you have at least 52% of free diskspace (see Conversion step 4).
- If you have enough diskspace, keep the old format recordings and delete them manually after checking the converted recording.
Conversion steps:
-----------------
Steps to convert the recordings (all after flashing the new image):
1) the box needs to be in stand by mode (WARM OFF), to get it there, press the power button on the remote briefly so that you get the powerdown menu (you might need to enable the menu if you have disabled it in the PGI settings). Choose the option Standby (the bottom one).
*** If the box isn't in standby mode, the conversion will fail!
If you have forgotten, the converter will automatically switch the box into standby mode before step 8.
2) The box will go into standby mode now.
3) telnet to your box
4) Check your free diskspace by typing:
'df'
and look for the line containing /dev/sda2
You need at least 1.5 times the free disk space as your longest recording. A value of 20 GB would suffice in most cases. If you plan to keep the old format recordings, you need at least 52% free disk space. The converter application doesn't check the available diskspace, so your are on yourself here, running out of diskspace during conversion will cause problems that looses the specific recording it is converting.
5) start the converter by typing:
'recconv'
(without the quotes and press enter)
6) The application will now ask you whether or not to keep the old format recordings, type either Y or N. Best would be to keep them after converting them, unless you plan to switch back to a previous (0.7.2a and before) version of PGI. Besides that, if conversion fails and isn't handled correctly, you can always rerun it
7) If you selected N at step 6, the converter will ask you to type the word 'delete' (without quotes) to make sure that you really want to delete them. Anything else then 'delete' will exit the converter.
The conversion will now start.
The box will automatically switch to warm off, then after 5 seconds the conversion will start (in case you wonder if anything happens, there is a 5 second delay before starting the conversion).
During conversion you can see the progress on the VFD.
9) If you want to break off the conversion (for example in the morning and the conversion isn't finished yet). Make a second telnet connection to your box and run 'stopconvert.sh' without the quotes.
The box will reboot now. The conversion can be restarted by starting at step 1 again any previous converted recordings will be skipped.
10) When the conversion is finished, the VFD will show Finished
11) When finished, the box will reboot itself automatically.
The conversion is now finished, after the reboot, your recordings will show up in the mediafile list (might be that they don't show up at opening the list the first time, but they will when closing the list and reopening it after a few minutes, populating the list is done in the background and might not have finished yet when you open it the first time).
If you have any questions (even before converting), please post them at denktenk.com
Manually deleting the old format recordings:
--------------------------------------------
If you decided to keep the old recordings and want to remove them after checking the converted recordings, you can do the following
telnet to your box
File by file:
WARNING: take a good look at the file name, old recording directory starts with ~ this is no typo!!! forgetting it will delete the new recording without a possibility to recover!!!!
cd /media
rm -r ~recordingname
rm recordingname.ifo
rm recordingname.dgsifo
rm recordingname.extinfo
You can also use the above to create diskspace
If you feel daring and want to to them all at the same time:
WARNING: take a good look at the commands below, the ~ is not a typo, leaving this one out erases you entire harddisk!!!!
cd /media
rm -r ~*
rm *.ifo
rm *.dgsifo
rm *.extinfo
Reconverting an already converted recording:
--------------------------------------------
If you find a recording that hasn't converted correctly (for example filesize is 0) you can do the following:
cd /media
rm -r name_of_recording
WARNING: the name_of_recording is the new one, don't delete ~name_of_recording as that is the old format version!!! The difference is the ~ this is not a typo.
Hard Disks, Flash Media, Formatting and Swap Space
===============================================================================
0.0 Storage device initialisation in PGI differs from other software images. The differences DO NOT AFFECT the devices in a standard configuration which is the following: ONE storage device (external USB or internal SATA) which has already been formatted by one of the DGS based images and was used for making the recordings before. If your configuration differs from the described or you are having troubles connecting some new device or you have custom partitions layout (for multiboot, skins or EPG storage only on USB sticks, etc.), then you should take your time and read the notes below.
0.1 PGI WILL NEVER FORMAT YOUR DISKS OR FLASH MEDIA BY ITSELF. Which means that if you want to connect new storage device like USB Flash or new SATA disk that has never been used before by the receiver, they will NOT be formatted and recognised automatically. This decision was deliberate and the main reason behind it was that while automated formatting may make life easier for some people, it may destroy valuable data for others who wish to use their connected devices for more than just pure PVR functionality. Since the software cannot magically guess what you intend to do with your storage device, you will have to choose device formatting option by hand when desired. Luckily, with PGI that procedure is quite straightforward.
0.2 In contrast with other images, the OSD menu called "HDD Settings" ALWAYS stays active. Therefore, if you want to format newly installed (or even an existing) device, you can do it at any time from that menu. Please pay attention to the message that is displayed once you choose the formatting option. The message will display the actual information about the device that will be formatted. If that device is not what you expect it to be, read the next paragraph.
0.3 What exactly will be formatted? This question may seem strange at first but only to those who have only one storage device connected. If you have more than one device connected (like SATA and USB together), then you have to be sure you are formatting the right disk. As a general rule, the disk which contains your /MEDIA partition will be formatted. Note that the whole disk with ALL(!) of its partitions be erased, not just the /media partition, so if you have non-standard disk layout with multiple partitions (used for multiboot, etc) you should never format your disk using receiver's OSD menu. In detail, the following occurs during format: first, ALL PARTITIONS will be deleted from the device that contains your /media partition and 2 new partitions will be created after that. The first partition (for example, /dev/sda1) will always be 256MB in size and will be used as a swap space (equivalent of the Windows' page file). The second partition (for example, /dev/sda2) will take the rest of the available disk space and will be used for the /media partition (i.e., PVR functions such as timeshift and channel recording). By default the ext3 file system will be used to format the /media partition. If you prefer ext2 instead, you can set the HDD_FS parameter in pgi.conf accordingly (see HDD_FS description below as well as the next paragraph)
0.4 Which file system is better: ext2 or ext3? The answer will be "it depends". ext2 performs slightly faster than ext3 which may become a crucial point for some devices with slower write speed. At the same time, ext3 handles errors much better since it is a journaling file system (and that makes it slightly slower). You have to try it for yourself. If your setup can handle ext3 it is better to stick with it rather than with ext2. If the write speed is not enough you will get "freezes" and "hiccupping" during timeshift and recording, especially with the high bitrate HD channels. For most SATA devices ext3 is fine.
0.5 Can I use another file system (not ext2 and not ext3) on my storage device? The short answer is no, not for recording. The longer answer is that you can only use ext2 or ext3 for recording but PGI images also understand FAT and NTFS file systems in the "read-only" mode, i.e. you can play media from USB flash formatted as FAT or connect your NTFS formatted HDD with the media files to view them but you will not be able to use devices with such format for channel recording. Why ext2 or ext3? Because they are native Linux file systems and are optimised for speed and error handling. For this very reason you cannot read the disks that you use for recording with Windows, if you decide to connect them to a PC later. There is some third-party Windows software available (some free and some not) to be able to read Linux file system like ext2 or ext3 under Windows. Why IPBox does not use more advanced Linux file systems such as ext4 or xfs? Because they require too much resources than what's available on IPBox and will be very slow or will not work at all.
0.6 If you have more than one storage device connected or you wish to use some non-standard disk layout with multiple partitions, you can select which partition should be used for /media via the MEDIA_DEVICE setting in pgi.conf. To make proper use of MEDIA_DEVICE setting it is mportant to determine the names of your storage devices. If you have only ONE device connected (does not matter if it's a SATA or a USB device) then that device will be named /dev/sda (and the partitions on it will be named /dev/sda1, /dev/sda2, etc.) If you have more than one storage device connected, SATA device will always have higher priority in terms of naming and usually will be called /dev/sda while any additional external USB disks will be then named /dev/sdb, /dev/sdc and so on. Finally, if you have several external USB devices connected at the same time, the order of their naming could change unpredictably from one boot to another (depending on how fast they reset during boot). In such (rare) situations you can use device's UUID (unique identifier) to specify the required partition for /media. To obtain UUID for all your existing EXT2/EXT3/FAT partitions you can execute "blkid" command in the Telnet session (also explained below in the MEDIA_DEVICE description).
0.7 PGI 1.x software uses most of the receiver's free RAM, especially when working with lots of EPG data. Therefore, it is highly recommended to have swap space enabled for the image to work properly. PGI will automatically detect all of the swap partitions that you may have and will make use of them without any additional configuration. For example, if you have internal HDD or external USB storage attached and formatted using receiver's OSD menu "Format HDD", then the swap space will be allocated as the 1st partition on the disk (and 2nd partition will be used as /media). If you have custom disk layout, you can allocate ANY partition for your swap space (you need to run mkswap command on that partition to mark it as swap) and it will be automatically detected and mounted during boot without changing any settings, no matter which partition it is going to be. Additionally, PGI image allows to use any ordinary file as a swap space. This means that if you want to have custom disk layout, you do not necessarily need to make a dedicated swap partition but can use an empty file on any other partition instead. If you decide to go ahead with a file as your swap space, you need to specify its full path in the SWAP_FILE parameter in pgi.conf (see its description below). WARNING: NEVER EVER use receiver's flash memory for swap, i.e. never assign files in /var directory for such purpose! It is also not recommended to use /media partition to hold your swap file because HDD health check that runs once in a while during boot may fail due to the fact that /media needs to be unmounted before checking (which means swap file cannot be used, which means you may run out of memory if the /media partition is too large in size).
0.8 If your receiver does not have ANY storage device attached internally or externally (usually the case with Cuberevo Mini/250/200 models), it is highly recommended to connect any USB stick (the faster speed, the better) and use it as a swap device. As an added bonus you will have possibility to store things like persistent EPG data, skins and picons on the same device. The USB stick does not need to be huge size if you are not planning to use it for channel recording; even 1GB stick would be more than enough to hold your swap space, EPG data, picons and skins. There are two possibilites with regards to how format such USB stick. One is to format it using receiver's "HDD Setup" menu, just like you would do it with a hard disk for PVR functions. In this case you will not need to change any default parameters in pgi.conf as both your swap and your /media partitions will be now on your USB stick (just don't forget to disable the Timeshift in the PVR options as soon as you have formatted the stick!) Alternatively, you can use ordinary FAT32 formatted USB stick with 1 partition on it, without reformatting it. In this case you need to change the following pgi.conf settings: SWAP_FILE, EPG_DATA_DIR and, if required, PICONS, SKIN and BACKUP_DIR to point to "/mnt/usb/sda1/...". The preferred way is the first one, because Linux native EXT3 file system will perform better than FAT32 in most scenarios.
0.9 To see the list of all partitions and/or files which mounted as swap you can use the following Telnet command: "cat /proc/swaps", to check the swap space usage you can use the following Telnet command: "free" (commands need to be typed it without quote marks). The "free" command shows more than just swap usage information, but you are only interested in the last line of the output that starts with "Swap:". It should contain three numbers. The first one is the total amount of swap memory (should be greater than 0 if swap is enabled) the second number is the amount of swap memory used at the moment (ideally it should be as low as possible), the last number is the amount of free swap space.
My Disk IS Not Being Recognised!
===============================================================================
1.0 You can always see some information about why your /media partition is not being recognised. The information is available in the "HDD Info" OSD menu and you should always start any troubleshooting from that screen. Most of the times it will be immediately apparent what went wrong just from the status message in that screen. If that information is not enough then you can enable debugging in pgi.conf (DEBUG="yes") and check the /var/log/hdd.log file after the reboot for most detailed information. Below you will find the description of most common errors and messages that you can see in the "HDD Information" menu.
1.1 Very often people think that their HDD has not been recognised, while actually it is simply being checked for errors during startup. The error checking may take a long time if you have many files (or many errors) on your disk and during this time the HDD will be "unavailable". You can see the disk checking progress in the "HDD Information" menu. The procedure of disk checking has 5 stages (and EACH stage goes from 0 to 100%). The disk checking can be enabled and disabled via the "HDD Settings" menu. It is a very good idea to check your disk once in a while to avoid data loss or corruption which could come from an unexpected reboot or panic. Generally speaking, the error checking is performed MUCH faster on ext3 file systems.
1.2 If you see the error that says "/dev/sda2 cannot be detected and no NFS is set" it means that the receiver cannot actually detect ANY storage devices connected to it (and you are not using NFS recording). There are 2 most common reasons for this:
1.2.1 If you get this message after you have installed new SATA disk, then chances are that the actual hardware is simply incompatible. Some people had success when they switched the HDDs from SATA 2 into SATA 1 mode by using a special jumper on their HDD.
1.2.2 If you get this message while using a USB HDD, the reason for the failure could be that it takes longer for the HDD to "spin up" after the cold start and the receiver checks for this device "too early". To overcome this problem you can force device detection (the device timeout will be longer) by setting HDD_USB_DETECT parameter in pgi.conf as follows: HDD_USB_DETECT="force" (see detailed description of this parameter below). If the device cannot be initialised even with such option, then you will see an error "/dev/sda detection timeout" in the "HDD Information" menu. In such case the only thing left to do would be to try another USB HDD model or provide external power to your USB disk if you have not done so.
1.3 If the error message says "/dev/sda2 does not exist", it means that your HDD *is* being detected, but is not partitioned the way IPBox expects it, i.e., second partition, is normally expected to be used for PVR tasks and it does not exist. There are two possible situations that may cause the error:
1.3.1 You device has never been used before for PVR functionality so it does not have correct partition layout. If you DO want to use this device for recording or timehift, you should simply reformat it using standard OSD menu option (see the beginning of this document).
1.3.2 You have a single USB device connected but do NOT wish to use it for PVR functions. Receiver would normally assume that the first detected device (sda) should be used for recording and will try to detect 2nd partition which is normally used for such purpose. Since in "receiver's opinion" your device is not partitioned the "right way" for PVR, you will get an error. In such case it is recommended to set the HDD_USB_DETECT parameter in pgi.conf as follows: HDD_USB_DETECT="no". This will prevent your receiver from trying to use your device for PVR functions.
1.4 If the error message says "/dev/sda2 has FAT file system" (or "NTFS file system" or "unknown system") that means that the device *is* detected, *has* the second partition, but the partition is not formatted using ext2 or ext3 file system. In such a case you need to format the device (see the beginning of this document).
1.5 If the error message says "/dev/sda2 cannot be mounted" that means that the device has correct format but due to some inconsistency or data corruption cannot be mounted. In such case it is best to attempt to run disk error checking by selecting this option from the "HDD Settings" menu and rebooting (you can watch the progress as described in 1.1 above). If that does not help, then reformat may be the only option.
1.6 The error "Cannot create tv/radio/multi dir" means that the device has been mounted, but no data can be written to it. Most often this happens with incorrectly set up NFS share used for recording when the /media device is mounted in a read only mode.
1.7 If the error message has something like "fdisk failed" or "mkfs failed" or "mkswap failed", that means that the HDD format was attempted but failed. You can try formatting one more time and if that still fails then it is worth checking the full log file as described in 1.0 above.
1.8 If the error is "NFS attempted but failed", then the connection to NFS recording share was attempted but failed. It is best to try to manually mount the same share from the Telnet session to see why it is not working. You should also check the correctness of your NFS path in MEDIA_DEVICE parameter (note that the colon sign is used twice for NFS paths).
Storage Device Related Parameters for pgi.conf
===============================================================================
In addition to the parameters listed in the REAMDE_en.txt file, there are some more HDD related options that can be set if required. It is best to only use them when you fully understand what you are about to do. As always, if the changes to these parameters cause non-booting receiver or data loss, it is your own problem.
HDD_AWAKE
By default, the shutdown sequence will try to spin down HDD shortly before shutting down power. If this feature is problematic for your configuration (some externally powered USB disks do not wake up until their power is reset), you can disable it by setting this parameter to "yes" in which case no command to spin down HDD will be sent.
Possible values: "yes", "no"
Default value: "no"
Example: HDD_AWAKE="yes"
HDD_FS
The name of the file system to use for /media partition formatting. Setting this option WILL NOT re-format the existing system and will not magically convert from one file system to another. You will have to manually invoke format from the OSD menu to change the file system (and lose all your data with it)! If this option is not set, "ext3" will be used the next time you format the storage device.
Possible values: "ext2", "ext3"
Default value: "ext3"
Example: HDD_FS="ext2"
HDD_SLEEP
The time in MINUTES after which the HDD should start sleeping, if inactive for that amount of time. "Inactive" means: there is no channel recording going on, timeshift is off and no EPG data, skins or picons is being accessed at the moment.
Possible values: 0 (no sleeping), 1 .. 20 and then 30, 60, 90, 120... etc.
Default value: "10"
Example: HDD_SLEEP="3"
HDD_USB_DETECT
Helps to solve problems with USB HDD detection in certain configurations. Normally receiver tries to autodetect external USB storage device automatically but sometimes autodetection does not work as expected. If your USB HDD does not get recognised at all during boot sequence you can try to set this parameter to "force" to enable alternative detection procedure. Also, if you have only 1 storage device permanently connected to your receiver but do NOT want to use it for PVR functions set this parameter to "no" to avoid false alarms related to "incorrect disk partitioning". However, for most configurations the "auto" setting works just fine. If you have problems recognising SATA drives, this is NOT the parameter that is going help and you should leave it at "auto" (see 1.2.1 above).
Possible values: "auto", "force", "no"
Default value: "auto"
Example (for problematic USB HDD that does not get detected nicely): HDD_USB_DETECT="force"
Example (if the *only* device is USB storage device and is *not* being used for PVR functions): HDD_USB_DETECT="no"
SWAP_FILE
Allows to specify a file that should be used as a swap space. The file MUST already exist (will not be created automatically) and it MUST be marked as a swap space. Both actions (creating an empty file of required size and marking it as swap) can be done using Telnet commands. For instance, the following commands will create 128 MB empty file in /mnt/usb/sda3 called pgi.swap and then it will be marked as swap space:
mkfile 128m /mnt/usb/sda3/pgi.swap
mkswap /mnt/usb/sda3/pgi.swap
Afterwards this file name can be specified as SWAP_FILE and it will be used IN ADDITION to all autodetected swap partitions on all connected disks. See 0.7-0.9 above for more information on swap space.
Default value: "none" (i.e. no special file used for swap, only the autotedected real swap partitions will be used)
Example (swap file on one of the partitions of an internal SATA disk): SWAP_FILE="/mnt/usb/sda3/pgi.swap"
Example (swap file on an external USB stick): SWAP_FILE="/mnt/usb/sdb1/myswap.tmp"
MEDIA_DEVICE
Allows to specify partition on SATA/USB device or NFS share which will be used as the /media folder (i.e. PVR functions such as channel recording and timeshift).
There are TWO possible ways to specify the SATA/USB device: using /dev/sdxX syntax and using device's UUID (see examples below). To see device UUID you can use 'blkid' command in the Telnet session.
There are TWO possible ways to specify an NFS share: when your NFS server uses UDP the path should start with nfs:// and when the server uses TCP the path should start with nfs-tcp:// (see the examples below).
A tip: you do NOT need to set NFSD="yes" in pgi.cong for NFS recording to work!
Default value: "/dev/sda2"
Example (SATA or USB storage device): HDD_MEDIA="/dev/sda3"
Example (SATA/USB storage device UUID): HDD_MEDIA="b7dfba7e-0e6b-4066-8906-d344e5a43912"
Example (NFS share using UDP protocol): HDD_MEDIA="nfs://192.168.1.2:/myshare"
Example (NFS share using TCP protocol): HDD_MEDIA="nfs-tcp://192.168.1.3:/myshare"
WARNING! If you change the default MEDIA_DEVICE setting and then format your HDD via the standard "HDD Setup -> HDD Format" menu, you have to remember 2 things:
1) Not only /media partition will be formatted, but the complete disk which contains media partition will be erased with ALL of its partitions!
2) The layout of such formatted disk will always be standard: 1st partition will be used for swap space, 2nd for media.
If you require different layout, never format your disk using the "HDD Setup -> HDD Format" menu and do it manually instead.
Enjoy your recordings,
PGI Team
