Windows MikTeX experience Today/LyX install

Posted on 2014/02/14 by pauljohn

The LyX-bundle 2.0.7 install was a thing of beauty. It "Simply Worked" for the most part. I was completely delighted. No stopping to ask the user about whether they want MikTeX, not mysteries.

Over the years, I have installed LyX for many Windows users and this was by far the best. Some folks in the LyX group have really invested a lot of effort to make this work well.

I think there are still problems where communication between LyX and MikTeX gets stuck, I'll describe below. For sure, I've learned that one should not have the MikTeX admin panels open when LyX tries to compile something. That's bad, since files are locked by one.

1. Toward the end of the install, a MikTeX updater panel appears and it asks which packages you want to install. We wondered why those packages were not just included in this "brand new" version of LyX. I know, just a detail

2. At the end, the green progress bar said we were at 100% and the installer spent quite a bit of time after that downloading LaTeX packages from a server in Europe. It didn't follow our request to use the American one in the MikTeX update chooser.

3. On the first document we tried to compile--the beamer template example--we got an error, the amsmath.sty file was missing. We used the MikTeX Maintenance(Admin) menu to go choose some packages. That seemed to fix it.

A little while later, I came back an tried same, and error message from MikTex was "Update helper could not be found". I googled way to a suggested fix. I a terminal (Start MS DOS Command box "as administrator") navigate to C:\Program Files (x86)\MikTeX\miktex\bin and run


mpm --admin --verbose --update

Note, these are double dashes, although the blog software makes them look single. It appeared that this fixed the problem the first time I came to use the MikTeX GUI updater, but just now I've tried again and got the update helper error. I'm almost ready to download a new copy of miktex-2.9 and install it to see if it adds some component that I've apparently damaged.

A similar problem happened when a user tried to compile the LyX APA6 template document. Error was that etex.sty was missing. We compiled again and the document succeeded

I suspect the following happened. LyX triggered the package installation behind the scenes? I notice now in MikTeX settings, the "install on fly" option is checked. I should have left it at that.

I caused a wild goose chase. I went into the MikTeX package manager and manually installed "miktex-etex-base". I did not read any documents, just installed it. After that, nothing at all complies in this system. I gather now that etex is extended LaTeX, and once that is installed, then base classes like "article.cls" are not found anymore. So I'm removing miktex-etex-base to see if I can make LyX basics work again.

Recall that LyX suggests the MikTeX "Update on the Fly" option. I think that makes for hangups of non-admin users who are not allowed to install packages.

As I close this note, I'm leaving this document uncompiled, and feeling sorry I went into MikTeX and added packages. The symptom I see is that "View PDF" in LyX causes pdftex to run, you can see it in the bottom bar in LyX. But no pdf ever appears for viewing. And I can't close LyX, error says "LyX could not be closed because documents are being processed by LyX."

Students have described this to me, but I'm not sure I'd ever seen it quite like this before. Maybe a system restart will help.

#####Update 2014-02-17

I've just been testing this on more windows computers. On a fresh LyX install, the first compiles take a really long time because LyX is trying to get MikTeX to install desired packages.

IF (only IF) I am logged in with administrative powers, then the work gets done, more or less in a hidden way. You can see it if, in LyX, you open the View Messages window when you try to compile the document. I just compiled the Beamer template document for the first time. Here's what I see:

Running: pdflatex  "newfile4.tex" > nul
17:44:43.106: 
17:44:43.106: ======================================================================
17:44:43.106: starting package maintenance...
17:44:43.106: installation directory: "C:\Program Files\MiKTeX"
17:44:43.106: package repository: http://ftp.fernuni-hagen.de/ftp-dir/pub/mirrors/www.ctan.org/systems/win32/miktex/tm/packages/
17:44:43.106: lightweight database digest: b5794978d47214eb6d50dd647fd52e29
17:44:43.106: going to download 168929 bytes
17:44:43.122: going to install 22 file(s) (1 package(s))
17:44:43.122: downloading http://ftp.fernuni-hagen.de/ftp-dir/pub/mirrors/www.ctan.org/systems/win32/miktex/tm/packages/sansmathaccent.tar.lzma...
17:44:43.122: 168929 bytes, 104.74 KB/Sec
17:44:43.122: extracting files from sansmathaccent.tar.lzma...
17:44:43.137: ======================================================================
17:44:45.852: 
17:44:45.852: ======================================================================
17:44:45.852: starting package maintenance...
17:44:45.852: installation directory: "C:\Program Files\MiKTeX"
17:44:45.852: package repository: http://ftp.fernuni-hagen.de/ftp-dir/pub/mirrors/www.ctan.org/systems/win32/miktex/tm/packages/
17:44:45.867: lightweight database digest: b5794978d47214eb6d50dd647fd52e29
17:44:45.867: going to download 275811 bytes
17:44:45.867: going to install 10 file(s) (1 package(s))
17:44:45.867: downloading http://ftp.fernuni-hagen.de/ftp-dir/pub/mirrors/www.ctan.org/systems/win32/miktex/tm/packages/filehook.tar.lzma...
17:44:45.883: 275811 bytes, 166.06 KB/Sec
17:44:45.883: extracting files from filehook.tar.lzma...
17:44:45.883: ======================================================================
17:44:49.627: Trying to make PK font ecss1000 at 600 DPI...
17:44:49.627: Creating ecss1000.pk...
17:44:49.627: Running miktex-mf.exe...
17:44:49.721: This is METAFONT, Version 2.718281 (MiKTeX 2.9)

What's slow about that? well, first, it is using a server in Germany. I had told MikTeX to use the KU math department's server, but it doesn't.

If I were not the administrator, this would "hang" because MikTeX is set to "install requested packages on fly" but the user who lacks admin privileges is stuck. I've argued in the LyX community the right answer to set LyX to refuse packages and thus provoke an error when LyX tries to run. I'd much rather get a message saying "a package is missing" than wait indefinitely while LyX argues with MikTeX.

Sometimes, the effort to beautify and simplify things for users is worse than leaving things the old-fashioned, hard to use way! I realize that's hypocritical. If I really thought that, I'd prefer to prepare LaTeX documents with a flat text editor, rather than LyX.

Posted in Uncategorized | Leave a comment

Ever run

$ ls -la

and suffer with a white background and white letters that look invisible? Suffer no more. Ever have a dark terminal and directory names are dark blue or black? Stop gauging your eyes out.

Step 1. Run dircolors

$ dircolors
LS_COLORS='rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;
35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;
41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;
31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;
31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;
31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;
31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;
31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.ace=01;
31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;
35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;
35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;
35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;
35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;
35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;
35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;
35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;
35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;
35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;
35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;
36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;
36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;
36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:';

export LS_COLORS

That shows the colors currently set in the terminal.

Step 2

Copy that into the bashrc, edit it to have colors you like.

di is directory color

Posted on 2013/12/30 by pauljohn | 2 Comments

Debian Wheezy Install Notes

Posted on 2013/11/17 by pauljohn

Install from netinst disk. Seems OK, but can't start session
using old home directory because that assumed I had compiz available.
And XFCE4.

Thus I have to do some unusual steps to get them, most users won't
need to bother because they will just let themselves be stuck with
Gnome 3. Awful. Awful.

1. XFCE4.

Alt-Ctl-F2 log in as root, use synaptic install:

xfce4 xfce4-goodies and everything they bring.

2. I must have compiz. It is a horrible travesty that compiz was removed from
Wheezy. Compiz now still available in sid.
Linux not fun without it. So I enable sid temporarily.

$ cat /etc/apt/sources.list.d/sid.list
deb http://ftp.us.debian.org/debian/ sid main non-free contrib

From there, I got compiz-8.4. Get ccsm as well! That
package is called "compizconfig-settings-manager".

After that, DISABLE access to sid. We don't want any
more unstable than we absolutely need.
Put a pound sign on that line afterwards to disable sid access.

3. Install packages. mlterm. emacs. auctex. ess.
dia. inkscape. geeqie. xfig

4. truecrypt. Installed, enabled. Need that! http://www.truecrypt.org

5. add i386 architecture. Will need that for acroread
and codeweavers crossover office.

# dpkg --add-architecture i386

After that, many packages will have i386 and x86_64 variants
(Debian calls later amd64).

6. apt-get lyx:amd64

Still not sure when the amd64 specifier will be required.

Now I notice the kernel is 3.2 and frequency scaling does
not work. Had to use experimental kernel to escape that one
in past.

7. add this file

/etc/sudoers.d/sudoers

with content

%sudo ALL=(ALL) ALL

and add me to sudo group

# addgroup pauljohn sudo

8. Old Debian installation still exists on /dev/sda5. Mount that,
so I can read the configurations I used to have and like.

8a. copy repository files for CRAN, multimedia, virtual box.

8b. touchpad settings. Default touchpad is HORRIBLE!

Copy over old config, which was in xorg.conf.d. No such directory
exists on Wheezy release:. I guess I have to just gamble and create one.
I notice there is no xorg.conf file in /etc/X11, so I doubt this will
work without creating one.

$ sudo mkdir /etc/X11/xorg.conf.d

$ cat /etc/X11/xorg.conf.d/synaptics.conf
Section "InputClass"
Identifier "touchpad catchall"
Driver "synaptics"
MatchIsTouchpad "on"
# This option is recommend on all Linux systems using evdev, but cannot be
# enabled by default. See the following link for details:
# http://who-t.blogspot.com/2010/11/how-to-ignore-configuration-errors.html
# MatchDevicePath "/dev/input/event*"

Option "VertTwoFingerScroll" "1"
Option "TouchpadOff" "0"
Option "PalmDetect" "1"
Option "PalmMinWidth" "20"
# Option "PalmMinZ" "7"
Option "CornerCoasting" "1"
Option "CoastingSpeed" "20"
Option "CoastingFriction" "0"
Option "RightEdge" "1750"
# Option "JumpyCursorThreshold" "100"
# Option "FingerPress" "255"
# Option "MaxTapMove" "90"
Option "TapButton1" "1"
Option "VertEdgeScroll" "1"
Option "FingerHigh" "100"
Option "FingerLow" "85"
Option "TapButton2" "3"
Option "TapButton3" "3"
Option "YAxisMapping" "5 4"
Option "HorizScrollDelta" "0"
Option "VertHysteresis" "25"
Option "HorizHysteresis" "25"
EndSection

9. Crossover office. Here is where the Debian
multiarch installation scheme is tested to it fullest.
Can we "fool" a package that is expecting to find all
the stuff in the huge ia32-libs package? Check this out:

pauljohn@debian:Debian$ sudo dpkg -i ia32-crossover_12.0.0-1_amd64.deb
Selecting previously unselected package ia32-crossover.
(Reading database ... 217270 files and directories currently installed.)
Unpacking ia32-crossover (from ia32-crossover_12.0.0-1_amd64.deb) ...
dpkg: dependency problems prevent configuration of ia32-crossover:
ia32-crossover depends on ia32-libs; however:
Package ia32-libs is not installed.
ia32-crossover depends on lib32nss-mdns; however:
Package lib32nss-mdns is not installed.
ia32-crossover depends on lib32z1; however:
Package lib32z1 is not installed.
ia32-crossover depends on python-glade2; however:
Package python-glade2 is not installed.
ia32-crossover depends on lib32asound2; however:
Package lib32asound2 is not installed.

dpkg: error processing ia32-crossover (--install):
dependency problems - leaving unconfigured
Errors were encountered while processing:
ia32-crossover

pauljohn@debian:Debian$ sudo apt-get install -f
Reading package lists... Done
Building dependency tree
Reading state information... Done
Correcting dependencies... Done
The following extra packages will be installed:
esound-common freeglut3:i386 gcc-4.7-base:i386 ia32-libs ia32-libs-i386:i386 lesstif2:i386
lib32asound2 lib32nss-mdns lib32z1 libacl1:i386 libaio1:i386 libasound2:i386 libasyncns0:i386
libattr1:i386 libaudio2:i386 libaudiofile1:i386 libavahi-client3:i386 libavahi-common-data:i386
libavahi-common3:i386 libbsd0:i386 libc6:i386 libc6-i686:i386 libcaca0:i386 libcap2:i386
libcomerr2:i386 libcups2:i386 libcurl3:i386 libdb5.1:i386 libdbus-1-3:i386 libdirectfb-1.2-9:i386
libdrm-intel1:i386 libdrm-nouveau1a:i386 libdrm-radeon1:i386 libdrm2:i386 libedit2:i386
libesd0:i386 libexif12:i386 libexpat1:i386 libffi5:i386 libflac8:i386 libfltk1.1:i386
libfontconfig1:i386 libfreetype6:i386 libgcc1:i386 libgcrypt11:i386 libgd2-xpm:i386 libgdbm3:i386
libgl1-mesa-dri:i386 libgl1-mesa-glx:i386 libglapi-mesa:i386 libglu1-mesa:i386 libgnutls26:i386
libgpg-error0:i386 libgphoto2-2:i386 libgphoto2-port0:i386 libgpm2:i386 libgssapi-krb5-2:i386
libice6:i386 libidn11:i386 libieee1284-3:i386 libjack-jackd2-0:i386 libjbig0:i386 libjpeg62:i386
libjpeg8:i386 libjson0:i386 libk5crypto3:i386 libkeyutils1:i386 libkrb5-3:i386 libkrb5support0:i386
liblcms1:i386 libldap-2.4-2:i386 libltdl7:i386 liblzma5:i386 liblzo2-2:i386 libmpg123-0:i386
libncursesw5:i386 libnspr4:i386 libnspr4-0d:i386 libnss3:i386 libnss3-1d:i386 libodbc1:i386
libogg0:i386 libopenal1:i386 libp11-kit0:i386 libpam0g:i386 libpciaccess0:i386 libpng12-0:i386
libpopt0:i386 libpulse0:i386 librtmp0:i386 libsamplerate0:i386 libsane:i386 libsane-extras:i386
libsasl2-2:i386 libsasl2-modules:i386 libsdl1.2debian:i386 libselinux1:i386 libsigc++-2.0-0c2a:i386
libslang2:i386 libsm6:i386 libsndfile1:i386 libsqlite3-0:i386 libssh2-1:i386 libssl1.0.0:i386
libstdc++5:i386 libstdc++6:i386 libsvga1:i386 libsysfs2:i386 libtasn1-3:i386 libtdb1:i386
libtiff4:i386 libtinfo5:i386 libts-0.0-0:i386 libusb-0.1-4:i386 libuuid1:i386 libv4l-0:i386
libv4lconvert0:i386 libvorbis0a:i386 libvorbisenc2:i386 libvorbisfile3:i386 libwrap0:i386
libx11-6:i386 libx11-xcb1:i386 libx86-1:i386 libxau6:i386 libxaw7:i386 libxcb-glx0:i386
libxcb-render-util0:i386 libxcb-render0:i386 libxcb1:i386 libxcomposite1:i386 libxcursor1:i386
libxdamage1:i386 libxdmcp6:i386 libxext6:i386 libxfixes3:i386 libxft2:i386 libxi6:i386
libxinerama1:i386 libxml2:i386 libxmu6:i386 libxmuu1:i386 libxp6:i386 libxpm4:i386 libxrandr2:i386
libxrender1:i386 libxslt1.1:i386 libxss1:i386 libxt6:i386 libxtst6:i386 libxv1:i386
libxxf86vm1:i386 odbcinst odbcinst1debian2 odbcinst1debian2:i386 python-glade2 uuid-runtime
xaw3dg:i386 zlib1g:i386
Suggested packages:
ia32-libs-gtk ia32-libs-gtk-i386:i386 libasound2-plugins:i386 nas:i386 glibc-doc:i386 locales:i386
pulseaudio-esound-compat:i386 rng-tools:i386 libgd-tools:i386 libglide3:i386 gphoto2:i386
gtkam:i386 gpm:i386 krb5-doc:i386 krb5-user:i386 jackd2:i386 liblcms-utils:i386 libmyodbc:i386
odbc-postgresql:i386 tdsodbc:i386 unixodbc-bin:i386 libportaudio2:i386 libroar-compat2:i386
libpam-doc:i386 hpoj:i386 hplip:i386 libsasl2-modules-otp:i386 libsasl2-modules-ldap:i386
libsasl2-modules-sql:i386 libsasl2-modules-gssapi-mit:i386 libsasl2-modules-gssapi-heimdal:i386
python-gtk2-doc
Recommended packages:
uuid-runtime:i386 xml-core:i386
The following NEW packages will be installed:
esound-common freeglut3:i386 gcc-4.7-base:i386 ia32-libs ia32-libs-i386:i386 lesstif2:i386
lib32asound2 lib32nss-mdns lib32z1 libacl1:i386 libaio1:i386 libasound2:i386 libasyncns0:i386
libattr1:i386 libaudio2:i386 libaudiofile1:i386 libavahi-client3:i386 libavahi-common-data:i386
libavahi-common3:i386 libbsd0:i386 libc6:i386 libc6-i686:i386 libcaca0:i386 libcap2:i386
libcomerr2:i386 libcups2:i386 libcurl3:i386 libdb5.1:i386 libdbus-1-3:i386 libdirectfb-1.2-9:i386
libdrm-intel1:i386 libdrm-nouveau1a:i386 libdrm-radeon1:i386 libdrm2:i386 libedit2:i386
libesd0:i386 libexif12:i386 libexpat1:i386 libffi5:i386 libflac8:i386 libfltk1.1:i386
libfontconfig1:i386 libfreetype6:i386 libgcc1:i386 libgcrypt11:i386 libgd2-xpm:i386 libgdbm3:i386
libgl1-mesa-dri:i386 libgl1-mesa-glx:i386 libglapi-mesa:i386 libglu1-mesa:i386 libgnutls26:i386
libgpg-error0:i386 libgphoto2-2:i386 libgphoto2-port0:i386 libgpm2:i386 libgssapi-krb5-2:i386
libice6:i386 libidn11:i386 libieee1284-3:i386 libjack-jackd2-0:i386 libjbig0:i386 libjpeg62:i386
libjpeg8:i386 libjson0:i386 libk5crypto3:i386 libkeyutils1:i386 libkrb5-3:i386 libkrb5support0:i386
liblcms1:i386 libldap-2.4-2:i386 libltdl7:i386 liblzma5:i386 liblzo2-2:i386 libmpg123-0:i386
libncursesw5:i386 libnspr4:i386 libnspr4-0d:i386 libnss3:i386 libnss3-1d:i386 libodbc1:i386
libogg0:i386 libopenal1:i386 libp11-kit0:i386 libpam0g:i386 libpciaccess0:i386 libpng12-0:i386
libpopt0:i386 libpulse0:i386 librtmp0:i386 libsamplerate0:i386 libsane:i386 libsane-extras:i386
libsasl2-2:i386 libsasl2-modules:i386 libsdl1.2debian:i386 libselinux1:i386 libsigc++-2.0-0c2a:i386
libslang2:i386 libsm6:i386 libsndfile1:i386 libsqlite3-0:i386 libssh2-1:i386 libssl1.0.0:i386
libstdc++5:i386 libstdc++6:i386 libsvga1:i386 libsysfs2:i386 libtasn1-3:i386 libtdb1:i386
libtiff4:i386 libtinfo5:i386 libts-0.0-0:i386 libusb-0.1-4:i386 libuuid1:i386 libv4l-0:i386
libv4lconvert0:i386 libvorbis0a:i386 libvorbisenc2:i386 libvorbisfile3:i386 libwrap0:i386
libx11-6:i386 libx11-xcb1:i386 libx86-1:i386 libxau6:i386 libxaw7:i386 libxcb-glx0:i386
libxcb-render-util0:i386 libxcb-render0:i386 libxcb1:i386 libxcomposite1:i386 libxcursor1:i386
libxdamage1:i386 libxdmcp6:i386 libxext6:i386 libxfixes3:i386 libxft2:i386 libxi6:i386
libxinerama1:i386 libxml2:i386 libxmu6:i386 libxmuu1:i386 libxp6:i386 libxpm4:i386 libxrandr2:i386
libxrender1:i386 libxslt1.1:i386 libxss1:i386 libxt6:i386 libxtst6:i386 libxv1:i386
libxxf86vm1:i386 odbcinst odbcinst1debian2 odbcinst1debian2:i386 python-glade2 uuid-runtime
xaw3dg:i386 zlib1g:i386
0 upgraded, 159 newly installed, 0 to remove and 0 not upgraded.
1 not fully installed or removed.
Need to get 60.4 MB of archives.
After this operation, 184 MB of additional disk space will be used.
Do you want to continue [Y/n]? y

I'm going to find the Microsoft Office disk and install a clean version.

10. Acrobad reader. Still the only pdf viewer that can print dependably.
# cat multimedia.list
# Debian Multimedia (unofficial): Debian 6.0 (Squeeze)
deb http://www.deb-multimedia.org wheezy main non-free
deb-src http://www.deb-multimedia.org wheezy main non-free

# apt-get install acroread
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
acroread-data acroread-debian-files:i386 acroread-dictionary-en
acroread-l10n-en libatk1.0-0:i386 libcairo2:i386 libdatrie1:i386
libgdk-pixbuf2.0-0:i386 libglib2.0-0 libglib2.0-0:i386 libglib2.0-bin
libgtk2.0-0:i386 libjasper1 libjasper1:i386 libpango1.0-0:i386 libpcre3
libpcre3:i386 libpixman-1-0:i386 libthai0:i386 libxcb-shm0:i386
Suggested packages:
acroread-plugins:i386 mozilla-acroread:i386 acroread-l10n:i386
librsvg2-common:i386 gvfs:i386 libjasper-runtime libjasper-runtime:i386
ttf-baekmuk:i386 ttf-arphic-gbsn00lp:i386 ttf-arphic-bsmi00lp:i386
ttf-arphic-gkai00mp:i386 ttf-arphic-bkai00mp:i386
Recommended packages:
mime-support:i386 acroread-dictionary-en:i386 acroread-dictionary:i386
hicolor-icon-theme:i386
The following NEW packages will be installed:
acroread:i386 acroread-data acroread-debian-files:i386
acroread-dictionary-en acroread-l10n-en libatk1.0-0:i386 libcairo2:i386
libdatrie1:i386 libgdk-pixbuf2.0-0:i386 libglib2.0-0:i386 libgtk2.0-0:i386
libjasper1:i386 libpango1.0-0:i386 libpcre3:i386 libpixman-1-0:i386
libthai0:i386 libxcb-shm0:i386
The following packages will be upgraded:
libglib2.0-0 libglib2.0-bin libjasper1 libpcre3
4 upgraded, 17 newly installed, 0 to remove and 296 not upgraded.
Need to get 37.2 MB of archives.
After this operation, 103 MB of additional disk space will be used.

9. Back to CPU frequency scaling. CPUS stuck in 800MHA, even though
they say "ondemand" is the governor.

Soon to begin lengthy hokus pokus.

In past, I was running kernel 3.5 from Debian experimental.

On the system, as originally installed, the cpu frequency scaling
monitors show the CPU frequency is stuck at 800MHZ. I'm using the
XFCE4 CPU Frequency Scaling monitor applet, which has always worked
very well. "ondemand" is the performance profile.

That should be sufficient. But the cpus are locked at 800.

I've seen this before on this in Debian Squeeze. There are a lot of
moving parts involved in fixing this. I'm not sure what is the bare
minimum, and while trying to fix it, I experimented with various
settings. If I were starting this from scratch, or telling you what to
do, I would try the fixes in this order.

1. Simplest. Suppose your setup is good, but the settings are bad.

My install had impossibly high scaling thresholds.

As root, check it:

#cat /sys/devices/system/cpu/cpufreq/up_threshold
95

Holy cow. No wonder the frequency is locked at 800mhz. Manually
re-setting that to something more reasonable:

# echo -n 60 > /sys/devices/system/cpu/cpufreq/up_threshold

I expect that may work. I did not try it first, before that I tried this.

Now, how to make that permanent?

$ apt-get install sysfsutils

In /etc/sysfs.conf I put:
mode devices/system/cpu/cpufreq/ondemand = 644
##devices/system/cpu/cpufreq/ondemand/freq_step = 10
devices/system/cpu/cpufreq/ondemand/up_threshold = 45
devices/system/cpu/cpufreq/ondemand/ignore_nice_load = 1
devices/system/cpu/cpufreq/ondemand/sampling_down_factor = 10

I DOUBT that will work if you don't already have cpufrequtils
installed. In case you don't, follow step 2.

2. Install & configure cpufrequtils. See
http://wiki.debian.org/HowTo/CpuFrequencyScaling

cpufrequtils should have been installed by default, but on my pc it
was not with Wheezy Beta 4.

I copied the configuration file I had set up from the other system

# cat /etc/default/cpufrequtils
ENABLE="true"
GOVERNOR="ondemand"
MAX_SPEED="2710000"
MIN_SPEED="800000"

Those 2 steps solve the problem for me. However, while testing, I did
explore idea that the kernel itself is the trouble.

3. Get a newer kernel from Debian Experimental. This was necessary in
Debian Squeeze and Wheezy before. I do not think it is necessary now,
but the kernel-3.2 supplied with Wheezy Beta 4 gives off a lot of
warnings on my system. So I tried 3.7 from Debian experimental. I'll
keep comparing. The 3.2 kernel lacked some stuff that was required to
us the enterprise wireless, but 3.5 did have that, and I'm hoping
either or both work when I try them later.

12. More debian packages. tk8.5-dev tcl8.5-dev autotools-dev.

13. Can't install R from CRAN website. Dependency on liblzma2 which
no longer exists.

So get the R source and try to rebuild. Quick, hold your nose, we
are doing under again

cd to someplace writable
$ apt-get source r-core

$ cd r-base-2.15.2

$ dpkg-buildpackage -rfakeroot

Build fails, asking for a list of dependencies:

$ sudo apt-get install gfortran libblas-dev liblapack-dev bison libncurses5-dev libreadline-dev texi2html libbz2-dev libpcre3-dev libpng-dev libjpeg-dev libxt-dev libpango1.0-dev libcairo2-dev libtiff4-dev xvfb texlive-fonts-extra mpack
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'libpng12-dev' instead of 'libpng-dev'
Note, selecting 'libjpeg8-dev' instead of 'libjpeg-dev'
The following extra packages will be installed:
fonts-comfortaa fonts-freefont-otf fonts-freefont-ttf fonts-gfs-artemisia
fonts-gfs-complutum fonts-gfs-didot fonts-gfs-neohellenic fonts-gfs-olga fonts-gfs-solomos
fonts-inconsolata fonts-junicode fonts-linuxlibertine fonts-oflb-asana-math libbison-dev
libcairo-script-interpreter2 libglib2.0-dev libice-dev libjbig-dev libpcrecpp0
libpixman-1-dev libreadline6-dev libsm-dev libtiffxx0c2 libtinfo-dev libxcb-render0-dev
libxcb-shm0-dev otf-freefont texlive-fonts-extra-doc texlive-xetex ttf-freefont
Suggested packages:
bison-doc fontforge gfortran-multilib gfortran-doc libcairo2-doc libglib2.0-doc libice-doc
ncurses-doc libpango1.0-doc libsm-doc libxt-doc inews latex2html cm-super
The following NEW packages will be installed:
bison fonts-comfortaa fonts-freefont-otf fonts-freefont-ttf fonts-gfs-artemisia
fonts-gfs-complutum fonts-gfs-didot fonts-gfs-neohellenic fonts-gfs-olga fonts-gfs-solomos
fonts-inconsolata fonts-junicode fonts-linuxlibertine fonts-oflb-asana-math gfortran
libbison-dev libblas-dev libbz2-dev libcairo-script-interpreter2 libcairo2-dev
libglib2.0-dev libice-dev libjbig-dev libjpeg8-dev liblapack-dev libncurses5-dev
libpango1.0-dev libpcre3-dev libpcrecpp0 libpixman-1-dev libpng12-dev libreadline-dev
libreadline6-dev libsm-dev libtiff4-dev libtiffxx0c2 libtinfo-dev libxcb-render0-dev
libxcb-shm0-dev libxt-dev mpack otf-freefont texi2html texlive-fonts-extra
texlive-fonts-extra-doc texlive-xetex ttf-freefont xvfb
0 upgraded, 48 newly installed, 0 to remove and 0 not upgraded.
Need to get 235 MB of archives.
After this operation, 584 MB of additional disk space will be used.

[whew]
After that, r-base builds

13. Annoying problem. Session management. XFCE4 starts, and a voice says "Welcome to Orca". I don't want that, where did it come from?

The XFCE4 session thing (In Settings menu, "Session and Startup") in the Advanced tab says "Launch Gnome services on startup." In the past I really really needed that because gnome ssh key manager services were required.

However, this gnome-orca thing comes in with.

I read that gnome-session-properties was supposed to be able to stop some things from loading, but I seem to be stuck with that damn orca because it keeps talking.

I don't want to remove the gnome-orca package altogether, because removing that causes the system to want to remove gnome. gnome is a meta package that I don't want to remove.

14. Compiz and the background.

I got compiz working and its background manager and my scripts to automatically update the separate workspace backgrounds.

However, it seems like when I restart, the system tries to fight me and
it wants to jamb the ugly XFCE4 background back on top and it obscures
compiz.

PITA.

must kill kfdesktop.

Posted in Uncategorized | Comments Off on Debian Wheezy Install Notes

Legal characters in file names

Posted on 2013/11/15 by pauljohn

My email signature has my opinion:

Today's Advice: File names include letters, numbers, periods, dashes, and underscores. NEVER \ / ! * & ^ $ or ". And no spaces.

Spaces are bad, they make everything inconvenient. I'm not a big fan of ( and ) in file names, don't want to see [ or ], and, generally, wish they would just be letters, numbers, dash and underscore.

When I was an MS Dos user, of course, file names were limited to 8 characters and the 3 letter extension, there could be at most one period, no spaces. When the Mac was introduced, they made spaces and long file names appear sexy, and Windows users used to dream eagerly of the day when they, too, could make life inconvenient for Unix programmers by putting spaces in file names. Nothing in scripting is easier with spaces, some things are more difficult.

There's a Microsoft page about this, note how ugly their URL is because of unacceptable characters in their name:

http://msdn.microsoft.com/en-us/library/aa365247%28VS.85%29.aspx

Anyway, they say for Windows NTFS file systems, one can use any characters except:

The following reserved characters:

< (less than) > (greater than)
: (colon)
" (double quote)
/ (forward slash)
\ (backslash)
| (vertical bar or pipe)
? (question mark)
* (asterisk)

Its obvious why they don't want the * in there, right? I'm interested to see they ban the colon, that's new information.

In shell scripts, we can usually work around these problems, but I would rather not create a crisis by assuming that all scritp writers are clever. Several characters are used in scripts, and we'd rather not confuse ourselves by seeing them in file names. That's a good reason why not to put
|
[ ]
( )
*
&

in file names. I shutter to think what would happen if a user puts spaces on both sides of a period, actually. A period in regular expression means "any character". Think how exciting " . " would be in a file name.

In my script directory in my home account, I keep a script that can replace bad characters, including spaces. It uses a time-tested Pearl program which was distributed with Slackware Linux in the early days (now I call it rename-perl.pl to keep it separate from the horrible rename program they package up now. Honestly, some people .... 🙂 ).

Here is rename-perl.pl in all its glory. I think this goes back to the inventors of Perl, actually,

#!/usr/bin/perl
#
# rename script examples from lwall:
# rename 's/\.orig$//' *.orig
# rename 'y/A-Z/a-z/ unless /^Make/' *
# rename '$_ .= ".bad"' *.f
# rename 'print "$_: "; s/foo/bar/ if =~ /^y/i' *

$op = shift;
for (@ARGV) {
$was = $_;
eval $op;
die $@ if $@;
rename($was,$_) unless $was eq $_;
}

If you make that executable and put it in your path (say, ~/bin), then you could run this to replaces of all spaces with underscores in file names in a directory:

$ rename-perl s/\ /_/g *

That middle part is a sed script, the asterix is a file selector (anything).

My script cleanFileNames.sh does that recursively. I have lots of other fixes in there
for other characters I hate, but here's the part you'd need to just kill file spaces.

#!/bin/sh

dirname="."
###if command line argument, it has to be a directory

if [ $# -gt 0 ]; then
dirname=$1
fi

for i in 0 1 2 3 4; do
find $dirname -maxdepth $i -name "*\ -\ *" -exec rename-perl s/\ -\ /-/g {} \;
echo "$dirname"
done

for i in 0 1 2 3 4; do
find $dirname -maxdepth $i -name "*\ *" -exec rename-perl s/\ /_/g {} \;
done

##snip details

Things worth gazing at:

Spaces in Filenames and Why You Should Avoid Them on the Web
http://www.blackbaudknowhow.com/blackbaud-sphere/spaces-in-filenames-and-why-you-should-avoid-them-on-the-web.htm

What technical reasons exist for not using space characters in file names?
http://superuser.com/questions/29111/what-technical-reasons-exist-for-not-using-space-characters-in-file-names

Work the Shell - Dealing with Spaces in Filenames
http://www.linuxjournal.com/article/10954

How to Create Good Filenames for Your Web Pages
http://www.thesitewizard.com/webdesign/create-good-filenames.shtml

UNIX File Names
http://cmgm.stanford.edu/classes/unix/filenames.html

Posted in Linux | Tagged , | Comments Off on Legal characters in file names

MacBook Pro: Hey, Jewel, Take a Walk on the Wild Side

Posted on 2013/10/28 by pauljohn

I've never used a Macintosh. I did not appreciate the original Macintosh at all. I felt they were charging a price premium for a system that was not open to hardware or software from other vendors. I chose MS DOS instead, then Windows, then Linux. Perhaps Macintosh is a bad idea whose time has come.

I'm working on a book manuscript about the computing environment of academic research and I get a lot of questions about Macintosh. I have to address this by walking a mile in their shoes. And I am writing it out now so I don't forget.

I came into possession of a MacBook Pro that's about 15 months old. It had another person's finishing touches on it, so I learned how to reformat, partition the drive, and then got a clean install of Mac OSX Mavericks, which was just released 2 days before I got this thing.

Coming from Linux, I was confident I could make this work because Mac is a glossy GUI on top of a Unix system.

First, note the system environment. Run Terminal, here it is:

$ cat env-mavrk-fresh.txt
TERM_PROGRAM=Apple_Terminal
SHELL=/bin/bash
TERM=xterm-256color
TMPDIR=/var/folders/90/03n12b2s705cb_1gpyhmvcj40000gp/T/
Apple_PubSub_Socket_Render=/tmp/launch-kl50om/Render
TERM_PROGRAM_VERSION=326
TERM_SESSION_ID=7FC9E7F9-22FB-49AA-BD4B-9B45CAD79DDF
USER=pauljohn
SSH_AUTH_SOCK=/tmp/launch-WixVbS/Listeners
__CF_USER_TEXT_ENCODING=0x1F6:0:0
PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
__CHECKFIX1436934=1
PWD=/Users/pauljohn
LANG=en_US.UTF-8
SHLVL=1
HOME=/Users/pauljohn
LOGNAME=pauljohn
_=/usr/bin/env

That's encouraging. Still BASH. Note the PATH is familiar.

I still see trouble, but I'm making this work.

The system has some of the features I've always hated (HATED) about the Mac GUI. It is horrible that the window is disconnected from the menu bar that goes with it. Got 10 windows open? Want to close one? Drag mouse to click the window, then up across everything else to close the thing. Bad design. That's a design for the "one program at a time" era of the early 1990s. Or cell phones. I ahve lots of things open at once and navigating is much worse in a Mac than any other OS. Even Windows. You can't just hit File -> Close, you have to highlight a window. This means that using a Mac requires twice as much clicking as any other OS. Choose a window, then choose form the menu.

Another bad side effect of the design is that you are forbidden from moving a window up so that its top is out of view. Sometimes I need to type while looking at a statistical result that is in the bottom of a window frame and that's impossible on a Mac. In Linux window managers, I notice they are imitating this bad feature, they make it tougher to turn off the "Y constraint" (in compiz, for example).

There are lots of other things that work nicely enough. Here's one that is fun. You can easily put different desktop wallpapers on various workspaces. How to get there is not obvious, but if you Google hard enough, you can make it work.

First, what about X11 support in the Mac these days. Here's what happens.

You find a big X under Utilities

"An application has requested access to X11. Would you like to install X11 now?

X11 is no longer included with OS X. Apple continues to support the development
of X11 on OS X with the open source community. Clicking "Continue" will
take you to an Apple Knowledge Base Article which provides information
about installing X11.

Link points at XQuartz-2.7.4.dmg

Safari doesn't let me specify "ask me everytime" on where to save downloads.
Inconvenient.

Did some exploring on dmg file format.

Inside Xquartz-2.7.4.dmg, there is a file XQuartz.pkg

pkg is an XAR file

tar xzvf XQuartz-2.7.4.dmg

cd to .pkg file in there, the thing Payload is actually a gz file,
rename as *.gz, then do stupid minuet

gunzip Payload.gz
cpio -iv < Payload to extract it. Should learn to just view file list. Install: Aquamacs-emacs-2.5 From r-project.org's download page R-3.0.2.pkg gfortran-4.2.3.pkg Verified that opening a *.R file in Aquamacs will activate ESS. Can't figure out lots of other details in the setup. That gfortran page points to other things required to build R packages. http://R.research.att.com/libs/ Can't figure what to do with all those tar.gz files XCode from Apple store. See the advice page on att.com. It is apparent they and the R page are not well coordinated. Each points at the other as the newest thing. http://r.research.att.com/tools See: http://www.tux.org/mactex MacTeX.pkg Lyx-2.0.6+qt4.dmg See http://www.lyx.org Please note that you need to install with Control-click. Otherwise, Mac will reject you because this is not registered with the Mac company. Aquamacs will not start from Terminal. In Aquamacs, run Tools pull down to install command line tools Otherwise necessary to run aquamacs from full path. http://www.gimp.org shows 2 possibles. I choose http://sourceforge.net/projects/gimpsonosx/files/Gimp%20Mountain%20Lion/Gimp-2.8.6p1-mountainLion.dmg gimp-2.8.6 ###### Consider Fink installer in future ###### Need VPN access, KU wants Cisco AnyConnect, download at http://technology.ku.edu/kuanywhere/download anyconnect-macosx-i386-3.1.0.01065-k9.dmg ## Same issue as linux, no 64 bit client. On Linux I dropped Cisco ## switched to Open Connect. There is no way in hell I'm going to log into ku.edu and go through the kuanywhere ritual every time I need a file. That's insulting. kuanwhere.ku.edu can be entered in the VPN line, one can log in. Weird, but the first time I run Cicso AnyConnect, it reaches KU and says it needs to install version 3.1.04066. please wait, over and over. Note that the KU RINGS system is incorrectly described in the ku pages. It does not check your computer. It checks your name. I don't mind Safari, actually. But I don't want to use a browser on only one platform. I read that I can transfer Mozilla profiles from my Linux system to this Macintosh. Firefox mozilla.org download: Firefox 24.0.dmg Thunderbird 24.0.1.dmg Those run from the Finder/Applications. So far, 2 most frustrating things about Mac are 1. It is not possible to do X11 style window move and resize. Can't click in middle of window and move with key & mouse combination. That's super inconvenient. 2. Most programs cannot be run from the Terminal without a battle. They are installed and linked to the Finder I don't mind Safari, actually. But I don't want to use a browser on only one platform. I read that I can transfer Mozilla profiles from my Linux system to this Macintosh. Other things are frustrating, it is tedious to get several terminals open side by side. Tough to cycle from one to other. Frustrating to move windows to new desktop Cant open terminal in one desktop if terminal already running in another. Trying to open terminal just snaps to other workspace where it is open. MUST have Truecrypt! www.truecypt.org/downloads Note that hits an install snag if you just double click through, it is like Windows saying the app is unapproved. So double click the package, it reveals a file "TrueCrypt 7.1a.mpkg". That will reject you if you double click. So Control-Single click (or other method of right clicking) and choose open. It will work after that. That's a problem because I don't really want a mac filesystem in a tryecrypt. The only alternative is FAT, which is bad for other reasons. Look for sshfs support http://osxfuse.github.io Package we are aiming for is sshfs, osxfuse is a pre-cursor osxfuse-2.6.1.dmg SSHFS-2.4.1.dmg In osxfuse, I chose option to turn on the gui interface to osxfuse config and update. We'll see what problems that causes. SSHFS runs into same permissions problem because it is from an uknown developer, so use control-click trick. For Secure FTP, I looked at various. Filezilla is familiar. http://filezilla-project.org But I was relly tempted by the Ducky http://cyberduck.ch The image viewer Preview in Mac is not great, can't cycle through images quickly. I'm looking for alternatives. Word is that Xee is ok. At least it has fast cycling through images. http://xee.c3.cx That is a trial version, may cost me $3.99 to keep using it. Solved one really irritating feature of the Mac. I've found how to create multiple desktops, but I hate this feature. Open a terminal on workspace 1. Go to workspace 2. Click terminal in the dock to open a new one. WHAM. Mac sends you back to workspace 1. In OSX 10.9, Mavericks, there is a setting to change. Click the apple top left, then System Preferences, then Mission Control. I unclicked the 2nd and 3rd buttons. Especially, Unchoose "When switching to an application, switch to a Space with open windows for the application". I also turned off "Automatically rearrange Spaces based on most recent use." I was really frustrated that workspace 2 was not "staying put". Maybe that will help. Oh, well. That was a good try. It worked for 10 minutes. Now system changed back. The Xee image viewer can't send a program to Gimp to edit, that's crappy. Googling for "GQView for Macintosh" I found this project MacPorts: Http://www.macports.org. I think I'll take the plunge, see what happens. MacPorts-2.2.1-10.9-Mavericks.pkg. I watched that a long time. It's compiling a linux distribution worth of tools. GTK, everything. I tried to install Gimp, it failed on aalib, claimint that malloc.h was missing. Quick google indicates this has been an intermitent problem. One bug report suggests it is necessary to run: $ sudo xcode-select --install I think I did that already, lets try again. After that, aalib compiles, so far as I can tell. It looks to me like Macports is the equivalent of Cygwin or such, which build unix tools on Windows. These things are not native, after installation. But I've not found the equivalent of cygwin.dll or such to glue them with the OS, either. So perhaps they are more like the GNUWin32 tools. Install "caffeine" from App Store (free) Why? I started a long job and then machine turned itself off while it was doing calculations. iTerm2: a terminal program alternative. Supposed to have more Unix-alike select & paste behavior, customizable background images. http://www.iterm2.com/#/section/downloads Installing iTerm: copy to Applications folder. Appears no way to install for one user only? Must be all? No HOME/bin or such? Spectacle: better window manager. That's encouraging http://spectacleapp.com ##################################### Aquamacs slow scrolling made me wonder about the "real Emacs" and I was pointed to Vincent Goulet's website http://vgoulet.act.ulaval.ca/en/emacs/ Emacs-24.3-modified-3.dmg ################################### RStudio RStudio 0.97.551.dmg ################################### KeePassX password safe KeePassX-0.4.3.dmg Supposed to be compatible with my Linux KeePassX file. ################################## gfxCardStatus for monitoring the graphics card http://gfx.io That doesn't seem to install, its an executable you run, from wherever. It indicates this MBP has no discrete graphics card. I'm removing it. Actually, quitting from the "i" applet, removing file should be sufficient.

Posted in Uncategorized | Comments Off on MacBook Pro: Hey, Jewel, Take a Walk on the Wild Side

RPM updates for RHEL6/Centos etc

Posted on 2013/10/19 by pauljohn

Hello, Everybody

This is an update notice/reminder. I've uploaded new Emacs and ESS for RedHat or Centos 6. And I also ask for bug reports, because of a question I will explain at the end.

My goal: have reasonably up-to-date Emacs, ESS and LyX, my favorites. The other stuff is incidental to that. I'm not (anymore) trying to offer my own LaTeX distribution because the one in RHEL6 is new enough these days.

Here's the RPM repository folder.

https://pj.freefaculty.org/EL/6/x86_64/

And the files, with upload dates:

Oct 17 06:44 emacs-nox-24.3-12.pjku.x86_64.rpm
Oct 17 06:44 emacs-debuginfo-24.3-12.pjku.x86_64.rpm
Oct 17 06:44 emacs-common-ess-13.05-2el6.noarch.rpm
Oct 17 06:44 emacs-common-24.3-12.pjku.x86_64.rpm
Oct 17 06:44 emacs-24.3-12.pjku.x86_64.rpm
Oct 17 06:43 emacs-common-ess-13.09-1pjku.noarch.rpm
Oct 17 06:43 emacs-ess-13.09-1pjku.noarch.rpm
Oct 17 06:43 emacs-ess-el-13.09-1pjku.noarch.rpm
Sep 24 18:07 emacs-common-ess-13.05-3el6.noarch.rpm
Sep 24 18:07 emacs-ess-13.05-3el6.noarch.rpm
Sep 24 18:07 emacs-ess-el-13.05-3el6.noarch.rpm
Aug 17 17:31 lyx-common-2.0.6-1.el6.noarch.rpm
Aug 17 17:31 lyx-debuginfo-2.0.6-1.el6.x86_64.rpm
Aug 17 17:31 lyx-fonts-2.0.6-1.el6.noarch.rpm
Aug 17 17:30 lyx-2.0.6-1.el6.x86_64.rpm
Feb 4 2013 pybliographer-1.2.15-4.el6.noarch.rpm
Feb 4 2013 python-bibtex-1.2.5-5.el6.x86_64.rpm
Sep 27 2012 lyx-common-2.0.3-1.el6.noarch.rpm
Sep 27 2012 lyx-debuginfo-2.0.3-1.el6.x86_64.rpm
Sep 27 2012 lyx-fonts-2.0.3-1.el6.noarch.rpm
Sep 27 2012 mlterm-3.1.3-1.x86_64.rpm
Sep 27 2012 mlterm-debuginfo-3.1.3-1.x86_64.rpm
Sep 27 2012 openbugs-3.2.2-2.x86_64.rpm
Sep 27 2012 swarm-2.4.1-1nojava.el6.x86_64.rpm
Sep 27 2012 swarm-debuginfo-2.4.1-1nojava.el6.x86_64.rpm
Sep 27 2012 swarm-static-2.4.1-1nojava.el6.x86_64.rpm
Sep 27 2012 tex-simplecv-1.6-9.el6.noarch.rpm
Sep 27 2012 tex-simplecv-doc-1.6-9.el6.noarch.rpm
Sep 27 2012 emacs-24.2-4.el6.x86_64.rpm
Sep 27 2012 emacs-common-24.2-4.el6.x86_64.rpm
Sep 27 2012 emacs-common-ess-12.09-1.el6.noarch.rpm
Sep 27 2012 emacs-common-ess-5.14-1.el6.noarch.rpm
Sep 27 2012 emacs-debuginfo-24.2-4.el6.x86_64.rpm
Sep 27 2012 emacs-el-24.2-4.el6.noarch.rpm
Sep 27 2012 emacs-ess-12.09-1.el6.noarch.rpm
Sep 27 2012 emacs-ess-5.14-1.el6.noarch.rpm
Sep 27 2012 emacs-ess-el-12.09-1.el6.noarch.rpm
Sep 27 2012 emacs-ess-el-5.14-1.el6.noarch.rpm
Sep 27 2012 emacs-filesystem-24.2-4.el6.noarch.rpm
Sep 27 2012 emacs-nox-24.2-4.el6.x86_64.rpm
Sep 27 2012 emacs-terminal-24.2-4.el6.noarch.rpm
Sep 27 2012 jags3-3.2.0-16.2.x86_64.rpm
Sep 27 2012 jags3-debuginfo-3.2.0-16.2.x86_64.rpm
Sep 27 2012 jags3-devel-3.2.0-16.2.x86_64.rpm
Sep 27 2012 lyx-2.0.3-1.el6.x86_64.rpm
Apr 13 2011 openbugs-3.2.1-1.x86_64.rpm

Only for historical reasons, the ESS packaging splits the files in two, you need emacs-common-ess and emacs-ess (that's related to previous support for Xemacs, which I'm not trying to support). I did not re-build emacs-auctex because the version in the RedHat optional channel appears reasonable to me, but perhaps I'm wrong, you tell me.

You can download individual files and install, or configure a repository to use with yum.

My package signing key is:

https://pj.freefaculty.org/EL/PaulJohnson-BinaryPackageSigningKey

If you download that and put it in this directory:

/etc/pki/rpm-gpg

it will co-operate with the repo file (you can copy into /etc/yum.repos.d) that I use is here:

https://pj.freefaculty.org/EL/pjku.repo

it refers to the signature in that particular directory. I think if you directly import the key, it will also work:

$ rpm --import https://pj.freefaculty.org/EL/PaulJohnson-BinaryPackageSigningKey

I might be able to save you some hassle on RedHat systems. On RHEL6, on the other hand, the default install does not necessarily include the optional repository, and that is needed. You will know there is a
problem if you run this and only see one channel (note, there are two dashes here).

# rhn-channel --list
rhel-x86_64-workstation-6

See? Only one repo. I added the optional repo like so (note single dashes):

# rhn-channel -a -c rhel-x86_64-workstation-optional-6
# rhn-channel --list
rhel-x86_64-workstation-6
rhel-x86_64-workstation-optional-6

Without that, you can't get emacs-auctex, or some of the things needed for texinfo files.

This diversion is not necessary if you are using Centos, they don't subdivide the repositories in the same way.
Now, about the possible bugs in my packaging you might report.

I use ESS in only the simplest way, I don't understand the use of many of its features. I may not notice if my packaging omits an ESS file. This happened in summer, 2013. I received requests for ESS-13.05 and I wrapped that up with the same SPEC file as previous ESS, and I made a mistake. The ESS 13.05 added some files, which I neglected to include, so some things did not work. In particular, I was missing ESSR.R and a couple related to SAS. That was fixed in

https://pj.freefaculty.org/EL/6/x86_64/emacs-common-ess-13.05-3el6.noarch.rpm

Recently, a request arrived for ESS-13.09. My package works for me, so far as I can tell I've included everything. Since I run Debian on my own systems, I can do a file-check of the ESS files against Dirk E's deb packaging, which I regard as the canon. So far as I can tell, I have this right, but you will have to tell me if you use the ESS features that I don't.

Here's the combined file list of my new ESS packaging in case you want to double-check:

https://pj.freefaculty.org/scraps/ess-13.09-files-centos.txt

and for comparison purposes, the ess file list from Debian

https://pj.freefaculty.org/scraps/ess-files-1309-deb.txt

Posted in R | Tagged , , | Comments Off on RPM updates for RHEL6/Centos etc

Lyx Local Layout, markup for R code in documents

Posted on 2013/09/19 by pauljohn

While preparing book manuscript, I found the need to mark code of various styles, R functions, variables, and I needed this to be flexible and fixable later when publisher changed mind on style. Here's the secret recipe.

In Lyx Document Settings

1. In modules, turn on the module "Logical Markup".

Test that out, see what they intend. You can highlight things and with a right click choose a markup style.

2. The existing markup may not suit your needs, but there is a customization route. In Document Settings, there is a thing "Local Layout". In there, put markup that will work with the module. Here is mine

Format 7
InsetLayout CharStyle:Rinput
  LyxType               charstyle
  LabelString           Rinput
  LatexType             command
  LatexName             Rinput
  Font
      Family             Typewriter
  EndFont
  Preamble
      \newcommand*{\Rinput}[1]{{\ttfamily #1}}
  EndPreamble
End

InsetLayout CharStyle:Rfunction
  LyxType               charstyle
  LabelString           Rfunction
  LatexType             command
  LatexName             Rfunction
  Font
      Family             Typewriter
  EndFont
  Preamble
      \newcommand*{\Rfunction}[1]{{\ttfamily #1}}
  EndPreamble
End

InsetLayout CharStyle:Routput
  LyxType               charstyle
  LabelString           Routput
  LatexType             command
  LatexName             Routput
  Font
      Family             Typewriter
  EndFont
  Preamble
      \newcommand*{\Routput}[1]{{\ttfamily #1}}
  EndPreamble
End

After that, right click magic!

Now also working on how to automatic the markup of matrices in latex. Have done it in one document, but will come back here to write.

Posted in Uncategorized | Comments Off on Lyx Local Layout, markup for R code in documents

Build R RPMS on an older Fedora

Posted on 2013/08/11 by pauljohn

I posted this in R-sig-fedora, or at least I thought I did. But the message did not appear.

A user wanted an R-3.0.1 RPM for Fedora 17, which reached end of life. That wasn't being provided by Fedora or Epel because F17 had reached end of life. Here's my answer, of which I'm proud.

It is easy to build your own RPM for your system. I don't want to discourage your from updating, but it would not be hard at all. (I wrote out how a couple of years ago. https://pj.freefaculty.org/blog/?p=73. This is the old style way, but I'm old)

I have a Fedora 17 virtual machine, but only in 32 bit. I'm trying this now. R does not rely on cutting edge software, I'll be stunned if I can't build it in Fedora 17. F17 is still newer than EL6 in most things, after all.

This is an old VM, I did not start it for at least 6 months. Not completely up to date. The first attempt to rebuild R fails, it needs RPMs for libSM-devel, libICE-devel, libjpeg-devel, libpango-devel, libicu-devel, and about 20 others. I was little worried this will necessitate a complete update.

No, OK. I ran into a little wrinkle that it found an old version of the R package "compiler" and tried to use that, but after removing R, I got a clean build. Evidence:

https://pj.freefaculty.org/Fedora/17/i386/R

R-3.0.1-2.fc17.i386.rpm 06-Aug-2013 15:58 18K
R-3.0.1-2.fc17.src.rpm 06-Aug-2013 15:59 24M
R-core-3.0.1-2.fc17...> 06-Aug-2013 15:59 52M
R-core-devel-3.0.1-2..> 06-Aug-2013 15:59 91K
R-devel-3.0.1-2.fc17..> 06-Aug-2013 15:59 17K
R-java-3.0.1-2.fc17...> 06-Aug-2013 15:59 18K
R-java-devel-3.0.1-2..> 06-Aug-2013 15:59 18K
libRmath-3.0.1-2.fc1..> 06-Aug-2013 15:58 244K
libRmath-devel-3.0.1..> 06-Aug-2013 15:58 23K
libRmath-static-3.0...> 06-Aug-2013 15:58 127K

There's one long-ish patch of almost build, then it fails because it finds the old copy of R's compile. After removing old R, all good. But at the end you see the RPM pop out.

It runs, I installed it. I can't say for sure if any functionality was omitted because this particular vm lacked some other devel package. But it proves a point. I guess you can tell if you look at the build output. I did most of this work in an Emacs eshell, you can see errors and failures.

https://pj.freefaculty.org/Fedora/17/i386/R/shell-20130806.txt

The 64bit version is left as an exercise for the reader 🙂 Or somebody else who has a virtual machine for that. Or somebody who wants to incentivize me.

Although I changed to other linux systems, I still like RPM. The RPM system is a very well thought out thing. And I marvel once again at the autoconf magic of the R Core Team. I've never had patience to learn that, but I admire people who have.

Posted in R | Tagged , , | Comments Off on Build R RPMS on an older Fedora

Posted on 2013/08/10 by pauljohn

Nobody has a cuter dog that the Honey dog, who has just endured a second leg surgery.
OLYMPUS DIGITAL CAMERA

Posted in Uncategorized | Comments Off on

R Summer Camp 2013 Journal

Posted on 2013/06/03 by pauljohn

## Summer Camp Blog 2013
## Day 1

I forgot to mention:

The new copy of the "rockchalk" package is on our testing repository


install.packages("rockchalk", repos = "http://rweb.quant.ku.edu/kran", type = "source")

Version 1.7.90 is in "feature freeze" now, bug fixes only. No new functions, no new data. To see the beauty of it, run the install, then

library(rockchalk)
example(plotSlopes)
example(testSlopes)

Your eyes will dance with joy, I assure you!

Now, here are the points of interest that I collected in our first day of Stats Camp, 2013

## Paul Johnson pauljohn at ku.edu
## Pascal Deboeck pascal at ku.edu
## 2013-06-03

## R functions mentioned in lectures and
## other pithy observations

## 1. Assignment symbol < - (don't use =)

x < - 1 + 2

## 2. Print contents of x

x

## same as

print(x)

## 2. list objects in a session
ls()
## ls is unix "list files", so natural to write same in R.
## apparently ls identical to
objects()

## 3. get out of R

q()
## usually say NO on saving workspace when it asks,
## unless you want to inherit same objects next time.
## Thus can short-circuit
q(save = "no")


## 4. c means "concatenate" or "column vector"

x < - c(1, 2, 3, 4, 5)

## Group together things of different types, and the lowest
## common denominator is adopted. Sometimes convenient, sometimes bad.

y <- c(1, 2, 3, 4, 5, "a")
y

## You can see why that would be disastrous. Right?

## 5. Math

sin(x)
log(x)
x^2
sqrt(x)

## 6. Stat. Note return can be singular or vector

sum(x)
mean(x)
sd(x)
var(x)
min(x)
max(x)
range(x)

## 7. How many elements are in vector x

length(x)


## 8. Remove things. Erase x

rm(x)

## erase everything found by ls()

rm(list = ls())

## 9. sort.

sort(x)
sort(x, decreasing = TRUE)

## 10. Read help pages

help("sort")
## same as
?sort
##
help.search("sort")


####


#### Section 2 begins

## 11. t.test

t.test(x)
t.test(x, y)

t.test(x, y, var.equal = TRUE)

## 12. sample: choose from a collection with equal likelihood

sample(1:15, 1)

x <- c("bill", "willie", "gil")
sample(x, 1)
qsample(x, 2, replace = TRUE)


## 13. Comparisons

==
<
>
< =
>=

identical()
all.equal()

& and
&& and (puzzle, why two ?)
|
|| (puzzle, why two ?)

NEGATION with "!"

## 14 Membership:

## selection by subscript. Use [ for subsetting
## with logical argument

undergrad < - c(TRUE, FALSE, TRUE, FALSE)
grade <- c(88, 44, 99, 11)
grade[undergrad]


## Binary: %in%

x <- c("a","b","a","b", "c")
y <- c("z", "y","z", "x","a")
x %in% y

x[x %in% y]


## 15. Dates

## Just characters
x <- c("25-01-2009", "30-12-2008", "01-01-2003")

y <- as.Date(x, format = "%d-%m-%Y")
y

## y is now a date object, which some regression programs
## handle in a special way. It is not just characters anymore.
## date objects can be subtracted to get time between values.

sort(x) ## alphabetical

sort(y) ## date progression


format(y, format = "%m-%Y")
weekdays(y)


## 16. Attributes, quick description of a thing's structure

attributes(x)

attributes(y)

str(x)

str(y)


## 17. Special symbols

## Reserved values : TRUE, FALSE, T, F, NA, NaN
## NEVER USE THOSE as variable names

## DO use those as variable values.
## All can be used as  special symbolic values in variables.
## DO NOT put those in quotes, they are special symbols.

x <- c(1, 2, 3, NA, 5, NA)

## Symbols generally we would not need
## Inf for infinity

## Special pre-defined variables

pi


## 18. Packages:

## See what's installed

library()

##Install

install.packages(c("psych"), dep = TRUE)

install.packages("psych", dep = TRUE, repos = "http://rweb.quant.ku.edu/cran")

## check

available.packages()

## load package

library(MASS)

library(help = "MASS")


## example(function-name-here)

example(lm)

example(plot)


## 19. Where am I?

getwd()

setwd()

## use forward slashed

After Lunch


###################AFTERNOON#############

## Matrices


## 20. cbind, matrix. A vector is a COLUMN vector.  In math, a
## "vector" is assumed to be column vector. This appears as a row

scale1 < - c(1, 5, 6, 5, 2, 6)
scale1

scale2 <- c(2, 5, 6, 8, 4, 8)
scale3 <- c(4, 1, 7, 1, 2, 1)

## but it is really a column, because it can be "column" bound with cbind


cbind(scale1, scale2, scale3)

## The matrix

matrix( c(scale1, scale2, scale3), nrow = 6)

## Or use matrix function. Various ways to do it

scaleCat <- c(scale1, scale2, scale3)

myMat <- matrix(scaleCat, nrow = 6) ## assumes byrow = FALSE



## 21. Ingerrogate a matrix: find rows, column

dim(myMat)

## dim gives size, but it has a super secret power: it can
## reshape vectors into matrices, or so forth. 

dimnames()

colnames(myMat) ## gets column names

colnames(myMat) <- c("scaleA", "scaleB", "scaleC") #assigns

## could add rownames with rownames()

## dimnames() reveals rownames and colnames.

## We could also set names with dimnames(), that requires a 
## list structure as input


## Use bracket notation

myMat[ 1, ] ##first row

myMat[ , 1] ## first column

myMat[3:5, 2]  ## rows 3-5 in column 2

myMat[c(1,3,5), ]  ## take rows 1 3 5


## Can create temporary vector, sum that

x <- myMat[ , 1]
sum(x)
## or go direct (faster, more difficult to debug)

sum(myMat[ , 1])

## if matrix has colnames, can access by name

colnames(myMat) <- c("hello", "goodbye", "hola")


myMat[ , "hello"]

myMat[ , c("hello", "goodbye")]



## 21 load() retrieves a saved R "thing"
## whatever is saved in there will "plop" into the workspace

## save(my-R-object, file = "myRthing.RData")

## Custom in R: the saved thing should be suffixed ".rda" or ".RData".
## no other suffix is accepted on CRAN for save data objects with packages.

## Note: saveRDS may be more to your linking. It saves only one
## particular thing, but the retrieval allows us to name the thing
## we get back.

load("myRthing.RData")


## 22. Data Frame

## Make a data frame

id <- c(1001: 1040)
rt1 <- runif(40, 250, 500)
rt2 <- rt1* 0.2 + runif(40, 200, 450)
gender <- sample(c("M","F"), 40, replace = TRUE)

dat <- data.frame(id, rt1, rt2, gender)

head(dat)

colnames(dat)

colnames(dat) <- c("id", "thing1", "thing2", "sex")

head(dat)

## colnames is an example of the same name being both an
## accessor and a setter.  They are actually 2 different functions,
## but the R programmers have created a clever way so that we
## only have to remember one name for those 2 purposes.




## 23. Lists

## R functions often return lists. and we need to navigate them

## Here's the way I'd do a t.test to find out if men and women
## have different means for rt1:

myttest <- t.test(rt1 ~ gender, data = dat)

attributes(myttest)

## You see:
## $names
## [1] "statistic"   "parameter"   "p.value"     "conf.int"    "estimate"   
## [6] "null.value"  "alternative" "method"      "data.name"  

## $class
## [1] "htest"

myttest$p.value
## same as

myttest[["p.value"]]

## Why 2 brackets?  This gives a list with p.value as one element in it

myttest["p.value"]

## whereas former rips vector "p.vector" out of list altogether.

is.list(myttest["p.value"])

is.list(myttest[["p.value"]])



##########Final afternoon session


## 24. Descriptive information
## min()
## max()
## mean()
## sd()
## var()
## cov()



## note most allow na.rm = TRUE, but cov and cor do not
## because they are matrix oriented. Necessary to take
## explicit steps to delete some rows(with NAs)

## Everybody needs central tendency values: mean, median, mode.

mean(x)

mean(x, na.rm = TRUE)
median(x)

## mode function returns data type, not statistical mode.

## for a factor variable x, this creates a frequency table
## and then reports back on which is most common
which.max(table(x))

## see that on the iris data

data(iris)

head(iris)


summary(iris)

## We dance around in a circle to get the most frequently
## observed value. Another example below
which.max(table(iris$Species))

theModeOfSpecies <- names(which.max(table(iris$Species)))


sd(iris$Sepal.Length)


## Caution: want the mean for each column? Necessary to do this
## choose first 4 columns, then use apply to get mean of each

apply(iris[ , 1:4], 2, mean, na.rm = TRUE)


## or just run

rockchalk::summarize(iris)



x <- sample(c("A", "B"), 150, replace = TRUE)
is.factor(x)
x <- factor(x)

table(x)
x.mode <- names(which.max(table(x)))


library(psych)
describe(iris)
## competes with rockchalk::summarize() and many other function
## that enhance summary.data.frame()



## 25 Inferential

t.test()

## X is a matrix

cor(X)

cor.test(X[, 1])

## Note problem: cor.test won't apply to whole matrix
## like some other programs might.

## What to do ?

## package HMISC has rcorr for testing matrix.

## Am testing alternatives on base R.

## Hmm. THis works, but still just one by one.
cor.test( ~ Sepal.Length + Sepal.Width, data = X)


X <- iris[ , 1:4]
X <- as.matrix(X)

## fails:
##
apply(X, 2, cor.test)

## Becoming angry, will have to think. I'm pretty sure
## this will devolve into some tedious computing on the language
## where I create a "mix and match" of variables. LIke this

Xnames <- colnames(X)

expand.grid(Xnames, Xnames)

MM <- unique(expand.grid(Xnames, Xnames))

## Now iterate through rows. Note redundancy and silly correlations
## calculated

mapply(function(n1,n2) {cor.test(X[ ,n1], X[ ,n2])}, MM[ ,1], MM[ ,2])

## Well those are numbers, but not clean.
##


chisq.test()


## fisher's exact test, a hypo test for small N, to help
## when chisquare has small cell sizes
fisher.test(data)


## Analysis of variance

## f: effect size for ANOVA, f = sdmeans/ sderror
## Cohen suggest 0.10=small, 0.25 = medium
## for small effect size:
data(ToothGrowth)
power.anova.test(groups = 6,  n = 10, between.var = 1, within.var = 100)

power.anova.test(groups = 3, n = 20, between.var = 1, within.var = 100)


## Homogeneity of variance

library(car) ## levene's test

## Massage the data to get omnibus groups variable

## See the notes, I am not sure I'm tracking along with this
leveneTest()

mpj1 <- aov(len ~ as.factor(supp), data = ToothGrowth)

A1 <- aov(len ~ as.factor(supp) * as.factor(dose), data = ToothGrowth)

anova(A1)
Posted in Uncategorized | Comments Off on R Summer Camp 2013 Journal