Wednesday, September 13, 2017

How to make MP4 video files smaller

The important command for this is:

ffmpeg -nostdin -i "$ORIGINAL_FILE" -vcodec libx264 -crf 26 "$OUTPUT_FILE"

The crf factor can be higher, then the file size will be even smaller.

It is possible to process more files at once.

Let's find files that have bitrate more than 3000 kb/s (which means we have probably not processed them yet):

find . -name "*.mp4" | while read F; do if [ `ffprobe "$F" 2>&1 | sed -n 's/.*bitrate:[^0-9]*\([0-9]*\).*/\1/p'` -gt 3000 ]; then echo "$F"; fi; done | tee tmp_mp4_list

If we want to see some details like the bitrate and file size, we can add this info for us:

find . -name "*.mp4" | while read F; do BTR=`ffprobe "$F" 2>&1 | sed -n 's/.*bitrate:[^0-9]*\([0-9]*\).*/\1/p'`; if [ "$BTR" -gt 3000 ]; then echo "$BTR `ls -lh "$F"`"; fi; done | tee tmp_mp4_list_details

We can pass the output from the first find command to ffmpeg:

cat tmp_mp4_list | while read F; do F_ORIG=`echo "$F" | sed 's/\(\.[^.]*\)$/_orig\1/'`; F_NEW=`echo "$F" | sed 's/.mp4$/.MP4/'`; mv -i "$F" "$F_ORIG" && ffmpeg -nostdin -i "$F_ORIG" -vcodec libx264 -crf 26 "$F_NEW"; if [ ! $? -eq 0 ]; then echo "$F" "$?" >> tmp_errors; fi; done

It will rename the original file to file_orig.mp4. The new one will have an uppercase extension .MP4 so that we can recognize we dealt with it already. We do not add extra characters to not make the filename longer in order to save space on a mobile display. If an error occurs, the filename will be mentioned in the 'tmp_errors' file.



Wednesday, May 17, 2017

Recover deleted photos from SD card

My friend came back from Africa with all photos on his SD card deleted. He knew that it is technically possible to recover them if they are not overwritten by new photos so he stopped using the SD card until he gave the card to me.

I used the program photorec to recover the photos. He used them happily for presentations.

With modern phones like Android, if the photos are stored in the internal storage instead of the SD card, I do not know an easy way to recover them (unless the phone is rooted).


Thursday, February 9, 2017

Hard disk full on Ubuntu

If you encounter low disk space on Ubuntu (eventually on other Linux flavour), a useful program to find size of directories is baobab.

You might see that /lib/modules contains gigabytes of files related to old kernel versions. How to remove them? The answer is here:

apt install --auto-remove

Tuesday, January 31, 2017

Migrate a WordPress blog into HubSpot

There is a tutorial on how to migrate WordPress blog posts into hubspot. If anything more complicated is needed, e.g. to deal with language variants, here are some commands that will help to automate it. I am also offering the migration as a service.

The situation was that the original blog had language variants in the form of http://blog.company.com/2017/... (without any language specified) for English and http://blog.company.com/ru/2017/... for other languages (in this case, "ru" for Russian).

Therefore, we process English blog items separately:

{ xmlstarlet ed -d '//item[not(contains(link,"blog.avast.com/20"))]' all.xml; echo ""; } > all-en.xml

Other languages are processed automatically with the help of a loop:

for L in cs de es fr it ja pl pt-br ru tr uk ; do export F=all-$L.xml;  { xmlstarlet ed  -d '//item[not(contains(link,"blog.avast.com/'$L'/"))]' all.xml; echo ""; } > "$F"; sed -r -i "s:(<link.*blog.avast.com/)$L/:\1:g" $F; done

There are redirects to be set up in the HubSpot settings so that the old URLs are still accessible and they will point to existing articles (new locations). This helps to maintain ranking in search engines and does not break links from other sites.

# english without a cycle
L="" perl -ne 'if(/^http:\/\/blog.avast.com(.*),http:\/\/avast.hs-sites.com(.*)$/ && not ($1 eq $2)) { print "$1,$2\n"}' > redirects-languages.txt


# other languages
for L in de it fr ru pl pt-br cs es; do
perl -ne 'if(/^http:\/\/blog.avast.com(.*),http:\/\/avast.hs-sites.com(.*)$/ && not ("/'$L'$1" eq $2)) { print "/'$L'$1,$2\n"}' >> redirects-languages.txt
done


One may have to deal with CDATA:

CDATA tag replace in ViM:

:%s#\(<script.*\)// <!\[CDATA\[#\1#
:%s#// \]\]\]\]><!\[CDATA\[></script>#</script>#

When uploading new redirects (which is possible to do in bulk), we deleted the old redirects first. It is possible to automate it with an iMacros script:

VERSION BUILD=8940826 RECORDER=FX
TAB T=1
URL GOTO=https://app.hubspot.com/content/486579/settings/url-mappings
TAG POS=1 TYPE=SPAN ATTR=CLASS:dropdown-targetsettings-icon&&TXT:
TAG POS=11 TYPE=A ATTR=TXT:Delete
TAG POS=1 TYPE=A ATTR=ID:hs-fancybox-ok

With these commands, it was possible to migrate thousands of blog posts from WordPress into HubSpot.


Wednesday, February 19, 2014

badblocks: says it is not safe to run and stops

Symptoms:

When checking for bad blocks, following error is displayed:

# badblocks -nvs /dev/sdb
/dev/sdb is apparently in use by the system; it's not safe to run badblocks!

Solution:

  1. Check the mounts (/etc/mtab), unmount with umount
  2. Check the swaps (/proc/swaps), unmount with swapoff
  3. Check volume groups (/dev/mapper/...), deactivate with vgchange -a n

Thursday, March 1, 2012

Dual screen in Ubuntu Unity/compiz

Besides what is obvious from configuring Displays, there is a useful shortcut that I did not find anywhere documented:

Start+P

It cycles through different display configurations (for the built-in LVDS notebook screen and for the external display). Unity tends to forget the settings in some cases (suspending the machine, disconnecting from the display) and this is where the shortcut becomes useful.

All windows end up on the first virtual desktop though. I did not find a solution for that. One can set up rules in compiz (with the ccsm utility) so that specific windows appear on specific virtual desktops, appear maximized, etc, then noone can move them from that position.

Tuesday, November 8, 2011

Lotus Notes fixpack in .deb traditionally produces this error:
dpkg -i ibm_lotus_notes_fixpack-8.5.2.i586.deb
dpkg: error processing ibm_lotus_notes_fixpack-8.5.2.i586.deb (--install):
 parsing file '/var/lib/dpkg/tmp.ci/control' near line 2 package 'ibm-lotus-notes-fixpack':
 error in Version string '8.5.2-20110323.0837_FP2': invalid character in revision number
To circumvent this error, unpack the .deb archive with:
 
  1. dpkg-deb --extract which extracts the files in the package.
  2. dpkg-deb --control which extracts the meta-data.
Edit this metadata (/DEBIAN/controls) and remove the underscore from "_FP2". Then, rebuild the package with  dpkg --build .

Fixed!

Monday, November 7, 2011

Encrypted Ubuntu filesystem on a logical volume (without alternate install)

Ubuntu with LVM and an encrypted filesystem does not require the Alternative installation media. It can be done via the official one.

My setup is what is usually called full-encryption (although I would not call it full, because the /boot partition is not encrypted and thus it can be backdoored by someone who has physical access to the machine). The root filesystem is located on a logical volume and encrypted (unlike in other setups and how-tos where the whole LVM is encrypted). The boot partition is on the /dev/sda3 partition; it can boot other systems where I do not want to install LVM support.

If a newly installed system does not boot after a reboot and drops out into a busybox shell, it can be because the initramfs is unable to find root partition to mount and boot the rest of the system. This is what I have done to finalize the installation:

1. Boot another system (e.g. Ubuntu from the installation media)

2. Open the new (encrypted) target Ubuntu root filesystem
(cryptsetup luksOpen /dev/vg/new cryptroot)
Of course you have not given your volume group a name like "vg" and your logical volume the name "new". (If yes... Do you like that all volume groups on your and your friend's USB drives have the same name "vg"? And do you want to have a logical volume that is old for many years but it is called "new", because you do not want to rename it?)

3. Mount it
(mount /dev/mapper/cryptroot /mnt)
4. Mount the /boot partition.
(mount /dev/sda3 /mnt/boot)
 Change the device number accordingly. This is the unencrypted boot partition. Kernel image and initramfs (initrd) boot files are located here.

5. Chroot to it
(chroot /mnt)
6. Install LVM and cryptsetup
(apt-get install lvm2 cryptsetup)
Once you install these packages, they will update the initramfs automagically. That is why it is important to have the boot partition mounted.

7. Edit the file /etc/default/grub and add kernel command line options

(GRUB_CMDLINE_LINUX="root=/dev/mapper/cryptroot cryptopts=source=/dev/vg/new")

so that after propagating the changes (update-grub) the generated grub configuration file (/mnt/boot/grub.cfg) has the line

(linux..... root=/dev/mapper/cryptroot cryptopts=source=/dev/vg/new)

You can omit the first option if root= option gets added automatically. You could omit the latter one (until 12/2011 in Ubuntu 11.10), but if the boot drops you to a" busybox initramfs shell" with an error that it cannot find /dev/mapper/cryptroot, this might be the useful.



8. Edit /etc/fstab

(/dev/mapper/cryptroot  /      ext3    errors=remount-ro 0       1)
9. Enjoy!

You can comment and ask, maybe I forgot some steps that were also needed. Consider also encrypting swap.

Monday, June 20, 2011

Skype does not ring in Ubuntu

Skype does not ring... More precisely, all events that display notifications fail to play sounds, but the others work fine (e.g. testing the event of "Ringing out" plays the sound).

Solution: System -> Preferences -> Sound



Pull up the Alert volume like above.

Thanks goes to http://ubuntuforums.org/showpost.php?p=10084372&postcount=4

Tuesday, May 31, 2011

OpenVAS says 'login failed'

This solution was found at

http://lists.wald.intevation.org/pipermail/openvas-discuss/2008-August/000732.html



"openvas-adduser" probably created the user directory in a different location (e.g. /usr/local/var/lib/openvas/) than the server is looking for it. It needs to moved to /var/lib/openvas.

Saturday, April 16, 2011

VirtualBox: Could not find an open hard disk with UUID

My VirtualBox sometimes shows a dialog box that says:

"One or more virtual hard disks, CD/DVD or floppy media are not currently accessible."

My virtual machine becomes inaccessible. Details show this error:

Could not find an open hard disk with UUID {5bad2863-64d0-40a2-a394-7442e3a6adc7}.
Result Code: VBOX_E_OBJECT_NOT_FOUND (0x80BB0001)

This happens when a snapshot is removed, then another snapshot is made and the host computer crashes (e.g. freezes during hibernation). After this, there is a reference to a non-existing HardDisk in the configuration file "~/.VirtualBox/VirtualBox.xml". This needs to be updated with the real value. The last working situation can be found in the log file (e.g. "~/.VirtualBox/Machines/<...>/Logs/VBox.log")

00:00:01.330 [/Devices/piix3ide/0/LUN#0/AttachedDriver/Config/] (level 6)
00:00:01.330   Format   = "VDI" (cb=4)
00:00:01.330   Path     = "/data/vmware/winvm/{5bad2863-64d0-40a2-a394-7442e3a6adc7}.vdi" (cb=62)
00:00:01.330   Type     = "HardDisk" (cb=9)
00:00:01.330
00:00:01.330 [/Devices/piix3ide/0/LUN#0/AttachedDriver/Config/Parent/] (level 7)
00:00:01.330   Format   = "VDI" (cb=4)
00:00:01.330   Path     = "/data/vmware/winvm/{cd71486f-ab71-4b03-9e2e-690d7ef76b9d}.vdi" (cb=62)
00:00:01.330
00:00:01.330 [/Devices/piix3ide/0/LUN#0/AttachedDriver/Config/Parent/Parent/] (level 8)
00:00:01.330   Format   = "VDI" (cb=4)
00:00:01.330   Path     = "/data/vmware/winvm/Windows XP flexible disk.vdi" (cb=48)


This shows reliably the structure (the snapshot and its parents are listed), so it can be reconstructed by editing the XML file accordingly.

Tuesday, January 18, 2011

asm/io.h: No such file or directory

The issue is caused by an include:

#include "asm/io.h"

Solution: Make a symlink asm -> asm-i386

Thursday, January 6, 2011

Fwd: firewire_core: Unsolicited response (external HDD drive & firewire)



MyBook connected through FireWire gets offline or has problems immediately after connecting:

Wednesday, November 24, 2010

How to download and compile mutt from source in Debian

apt-get source mutt
    - nebo stáhnout

apt-get build-dep mutt
    - nainstaluje balíčky podle závislostí,
      aby šlo instalovat source balík mutt

    - kdyby to nešlo (chyba "Build-dependencies for rpm could not be satisfied"):
        - přidat za příkaz: -o Debug::pkgProblemResolver=true

configure-optiony se dávají do debian/rules

dpkg-buildpackage -rfakeroot -us -uc

(dělal jsem to pod rootem, nevím, proč mi druhý pokus selhal - možná jsem neaplikoval patch, už ani nevím, kde se to provádí - z rozbaleného tar.gz se druhý pokus nepovedl)

* kam som dal prepínač --with:
   do debian/rules

    * v gdb:
        v debian/:
        echo decstation-ultrix >target

Monday, February 22, 2010

Dell Latitude volume keys under Debian

Dell laptops have volume keys above the F-keys. This post explains how to enable them to control the volume.

First, the keys need to be recognized. One can set the right keyboard model in /etc/default/console-setup:
XKBMODEL="latitude"
Other options can be set also, e.g.:
XKBLAYOUT="us,cz"
XKBVARIANT=",qwerty"
XKBOPTIONS="lv3:ralt_switch,grp:alt_shift_toggle,grp:rwin_switch,grp:menu_toggle,grp_led:scroll"
That ensures that keys are mapped to names like "XF86AudioMute". Names can be used in the menu file, where you put commands for alsamixer to rise/lower the volume. Your menu file could be named .menu/volume-keys (explained in "Debian menu system"):
?package(alsa): needs="X11" section="Apps/Sound/Xmms commands" \
        title="Master mute" \
        shortcut="XF86AudioMute" \
        longtitle="X MultiMedia System" \
        command="amixer set Master toggle"
?package(alsa): needs="X11" section="Apps/Sound/Xmms commands" \
        title="Master up" \
        shortcut="XF86AudioRaiseVolume" \
        longtitle="X MultiMedia System" \
        command="amixer set Master 5%+"
?package(alsa): needs="X11" section="Apps/Sound/Xmms commands" \
        title="Master down" \
        shortcut="XF86AudioLowerVolume" \
        longtitle="X MultiMedia System" \
        command="amixer set Master 5%-"
Run update-menus and refresh your window manager. Finished.

By choosing the right keyboard model, following lines are applied, which should already be present somewhere in /usr/share/X11/xkb/symbols:
    key    { [ XF86AudioMute         ] };
    key    { [ XF86AudioLowerVolume  ] };
    key    { [ XF86AudioRaiseVolume  ] };

Saturday, June 13, 2009

Firefox displays blank Alert and ssl is disabled

I've got error "ssl_error_ssl_disabled" (Firefox cannot connect securely because the SSL protocol is disabled).

During start, there was always a blank / empty Alert box. That was probably "Could not initialize the browser's security component."

Solution: Moved cert8.db file from another profile (you can also delete it).

Friday, March 6, 2009

WindowMaker launching another instance from dockbar

For launching another instance of an application by clicking on the icon in the dock, hold down Ctrl.

I have two xterm apps in the dockbar, one for a todo-list and one for "done" (aka knowledge base):

xterm -class UXTerm -title done -u8 -g 50x40 -e vim /z/txt/txtcvs/done -c $

xterm -class UXTerm -title todo -u8 -g 50x40 -e vim -O /z/txt/txtcvs/todo -S /z/.vim/todo-macros.vim


If xterm is running, it is impossible to launch "todo" without Ctrl. Or you can run more instances of e.g. Firefox by this...

Monday, March 2, 2009

nmap script engine aborting

Nmap 4.68 from debian repository is missing required files.

Workaround: Download the tarball and copy nselib/ to /usr/share/nmap, as written here:

http://www.nabble.com/Bug-495054:-module-'comm'-not-found-td18980885.html

Wednesday, February 11, 2009

cron not running commands

crontab was not working. Silently, not running the command, not writing anything to /var/log/cron. You can check the syntax of your crontab using:

# crontab /etc/crontab "/etc/crontab":16: bad minute errors in crontab file, can't install.

The problem was in splitting command into more lines using backslash \ at the end of each line.

Simple workaround: Put it on one line.

Do not forget also that the minute column is right to the hour column.

Wednesday, January 28, 2009

jUnit / jMock: error in opening zip file

I was getting an exception using jUnit / jMock tests together with Ant:

  [junit] java.util.zip.ZipException: error in opening zip file
  [junit] at java.util.zip.ZipFile.open(Native Method)
  [junit] at java.util.zip.ZipFile.(ZipFile.java:114)
  [junit] at java.util.zip.ZipFile.(ZipFile.java:131)
  [junit] at org.apache.tools.ant.AntClassLoader.getResourceURL(AntClassLoader.java:1028)
  [junit] at org.apache.tools.ant.AntClassLoader$ResourceEnumeration.findNextResource(AntClassLoader.java:147)
  [junit] at org.apache.tools.ant.AntClassLoader$ResourceEnumeration.(AntClassLoader.java:109)
  [junit] at org.apache.tools.ant.AntClassLoader.findResources(AntClassLoader.java:975)
  [junit] at java.lang.ClassLoader.getResources(ClassLoader.java:1016)
  [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.checkForkedPath(JUnitTask.java:1126)
  [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.executeAsForked(JUnitTask.java:1013)
  [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.execute(JUnitTask.java:834)
  [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.executeOrQueue(JUnitTask.java:1785)
  [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.execute(JUnitTask.java:785)
  [junit] at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
  [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  [junit] at java.lang.reflect.Method.invoke(Method.java:597)
  [junit] at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
  [junit] at org.apache.tools.ant.Task.perform(Task.java:348)
  [junit] at org.apache.tools.ant.Target.execute(Target.java:357)
  [junit] at org.apache.tools.ant.Target.performTasks(Target.java:385)
  [junit] at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1337)
  [junit] at org.apache.tools.ant.Project.executeTarget(Project.java:1306)
  [junit] at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
  [junit] at org.apache.tools.ant.Project.executeTargets(Project.java:1189)
  [junit] at org.apache.tools.ant.Main.runBuild(Main.java:758)
  [junit] at org.apache.tools.ant.Main.startAnt(Main.java:217)
  [junit] at org.apache.tools.ant.launch.Launcher.run(Launcher.java:257)
  [junit] at org.apache.tools.ant.launch.Launcher.main(Launcher.java:104)




Solution:

Use

    <path id="classpath.run.tests"> 
        <pathelement location="." />                                                                 
    </path>


instead of

    <path id="classpath.run.tests">                                                                  
        <fileset dir=".">                                                                            
            <include name="*.class" />                                                               
        </fileset>                                                                                   
    </path>


The explanation.

Monday, March 26, 2007

Splitting vim vertically

You want vim to split vertically?

Solution:   :vsplit [filename]




Or you can run vim from command line with option -O,

example:
vim -O ~/todo ~/done

Thursday, March 22, 2007

CVS overwrites $Revision, $Id in my data

I imported my spam-filter project into CVS also with mails it is testing on.

It modified my mails undesirably, where "$Revision" or "$Id" occurred.

Solution: cvs import -ko

I typed cvs import -ko -m "Imported all files" z/rp z start, where the option -ko turns all substitions off.

More in http://ximbiot.com/cvs/manual/cvs-1.12.13/cvs_12.html#SEC102