Rare Events, Bias Corrected Logistic Regression, etc.

Posted on 2013/04/12 by pauljohn

I'm copying this to the blog so I can assemble answers here. I expect this could be interesting.

Dear R-sig-mixed:

I was struck today by the way the Internet has accelerated research. At one time, it might have taken a month or two to track down the articles on this problem and conclude I need to ask for advice. Now, however, I realize the need within hours.

Recall the question that started us debating a few days ago was a logistic regression in which OP noticed the mis-match between the predicted probability of success and the observed fraction.

(R-sig-mixed, 2013-04-03, OP: Zack Steel, "Low intercept estimate in a binomial glmm")

We were debating that, and it had completely slipped my mind that there is a separate literature on exactly that kind of problem. Yesterday, somebody else asked me to estimate a logit model in which there were more than 40000 cases but only a few hundred "successes". That's what reminded me of the "rare events" problem and logistic regression parameter estimate bias.

And I think that's the issue that we need to clear up with glmer. What do you think? Since multilevel model can be seen as a penalized ML estimation (ala Pinheiro and Bates, or as explained in Simon Wood, Generalized Additive Models), are we able to get a bias-corrected variant?

Furthermore, could lme4's predict method be made to produce "good" confidence intervals. And that leads down a separate path to a huge hassle about competing ways to estimate CI's in glm and the possible need to appy extra corrections in some special cases. I'll write down that problem to ask you about it later if you help me understand this one.

Here's my brief novel on what I've been Googling about for the past 10 hours or so. If it helps you, let me know. If you think I'm wrong, especially urgently let me know.

To the political science audience, that's a "rare events" logistic regression problem, our most heavily cited methods paper on that is:

King, G., & Zeng, L. (2001). Logistic Regression in Rare Events Data. Political Analysis, 9(2), 137–163.
http://pan.oxfordjournals.org/content/9/2/137.abstract

Logistic parameter estimates (mainly the intercept) are wrong and estimated probabilities are wrong. King & Zeng provided Stata code for a function "relogit" and later adapted same for R (package: Zelig). Zelig tries to re-organize the whole regression experience for the R user, and I didn't want that, so I started looking into the various corrections to see if I couldn't write an adapter to take a glm or a glmer output and "bias correct" it. It appears, superficially at least, that I only need to adjust the intercept estimate by a weighting factor, which would be super easy to do.

Quite by chance, I found this blog post by Paul Allison, and its really interesting!

Logistic Regression for Rare Events (2012-02-13)
http://www.statisticalhorizons.com/logistic-regression-for-rare-events

And, wow, is it subtle. Read that over a few times, see if you agree with me. In a kind way, he says the "rare events" business is a red herring, and instead we need bias-corrected logistic regression estimates. Use David Firth's method. The part about the "prior correction of the intercept" discussed in King and Zeng, is not the best approach. Instead, we should see this as a symptom of the more general problem that ML estimates are biased and the bias is greatest when there are not too many "successes". Allison suggests an estimator proposed by David Firth, which used penalized ML.

Firth D. Bias reduction of maximum likelihood estimates. Biometrika 1993; 80:27–38

I don't think King and Zeng disagree, they also propose an option to bias-correct the whole vector of coefficients. That bias correction ends up addressing the more general problem. In the Stata module for relogit (the version I found was dated 1999-10-28), it says ""Relogit for Stata does not yet support the FIRTH option", but it does have an alternative weighting correction.

While fiddling around to see if I could implement that, I learned it has been done in R:

logistf: Firth's bias reduced logistic regression

http://cran.r-project.org/web/packages/logistf/index.html

That is often discussed as a solution to the problem of separation, as on the UCLA stats website, (http://www.ats.ucla.edu/stat/mult_pkg/faq/general/complete_separation_logit_models.htm)

Georg Heinze and Michael Schemper, A solution to the problem of separation in logistic regression, Statistics in Medicine, 2002, vol. 21 2409-2419.

But it is a two-fer, so far as I can tell. We get bias correction and separation-proofness.

Heinze, G., & Puhr, R. (2010). Bias-reduced and separation-proof conditional logistic regression with small or sparse data sets. Statistics in medicine, 29(7-8), 770–777. doi:10.1002/sim.3794

##Additional information

Heinze and Ladner offer an R package "logistiX", for "exact" logistic regression.

The part I don't understand (yet) is how the bias correction links to mixed models. And that's why I'm asking you.

OK?

Tagged | Comments Off on Rare Events, Bias Corrected Logistic Regression, etc.

R Bioconductor Install (or, where’s my “graph” package?)

Posted on 2013/04/03 by pauljohn

Users reported that an R package I use, gRbase, "doesn't work". I had to watch them try it and read the error messages.

When you try

> install.packages("gRbase", dep = TRUE)

The packages "gRbase" depends on other packages, like "graph", and the messages tell you that those other things are unavailable. So the install fails. Then you try to install graph, and that fails. It says that there is no "graph" for R version 2.15.3.

Following up, I realized that "graph" is now on Bioconductor with some other really good packages. Bioconductor is a supplementary repository, one of the leaders in Bioconductor is Robert Gentleman, who was a founder of R.

How to install that? Read at

http://bioconductor.org/install

The simplest way is to rely on the script they wrote to configure everything automatically.

> source("http://bioconductor.org/biocLite.R")
> biocLite("graph")

Here's what I see:

> biocLite("graph")

BioC_mirror: http://bioconductor.org
Using Bioconductor version 2.11 (BiocInstaller 1.8.3), R version 2.15.
Installing package(s) 'graph'
trying URL 'http://bioconductor.org/packages/2.11/bioc/src/contrib/graph_1.36.2.tar.gz'
Content type 'application/x-gzip' length 941743 bytes (919 Kb)
opened URL
==================================================
downloaded 919 Kb

* installing *source* package ‘graph’ ...
** libs
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG      -fpic  -O2 -pipe -g  -c graph.c -o graph.o
In file included from graph.c:4:0:
/usr/share/R/include/R_ext/RConverters.h:32:2: warning: #warning "R_ext/RConverters.h was deprecated in R 2.15.1 and will be removed in R 3.0.0" [-Wcpp]
gcc -std=gnu99 -shared -o graph.so graph.o -L/usr/lib/R/lib -lR
mv graph.so BioC_graph.so
installing to /home/pauljohn/R/x86_64-pc-linux-gnu-library/2.15/graph/libs
** R
** data
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
   ‘GraphClass.Rnw’
   ‘MultiGraphClass.Rnw’
   ‘clusterGraph.Rnw’
   ‘graph.Rnw’
   ‘graphAttributes.Rnw’
** testing if installed package can be loaded

* DONE (graph)

The downloaded source packages are in
        ‘/tmp/RtmpqukmNQ/downloaded_packages’
Old packages: 'Amelia', 'ape', 'ash', 'BayesX', 'bdsmatrix', 'Boruta', 'caret',
  'clue', 'corpcor', 'dse', 'Ecdat', 'edgeR', 'eha', 'ENmisc', 'entropy',
  'FactoMineR', 'FAiR', 'ff', 'FitAR', 'flowCore', 'flsa', 'functional',
  'gdata', 'gtools', 'hexbin', 'isa2', 'klaR', 'kohonen', 'ks', 'laeken',
  'limma', 'lpSolve', 'mboost', 'mirt', 'miscTools', 'multcomp', 'mvabund',
  'np', 'pan', 'partykit', 'pcaPP', 'pequod', 'pracma', 'pspline', 'randomLCA',
  'raster', 'Rcpp', 'RcppArmadillo', 'RCurl', 'relations', 'rgeos', 'RJSONIO',
  'RWeka', 'RWekajars', 'sandwich', 'semTools', 'seriation', 'sets',
  'sgeostat', 'shiny', 'sp', 'spdep', 'statmod', 'svMisc', 'testthat',
  'tweedie', 'vegan'
Update all/some/none? [a/s/n]:
Update all/some/none? [a/s/n]: n
Warning message:
installed directory not writable, cannot update packages 'actuar', 'akima',
  'ape', 'arm', 'bdsmatrix', 'Biobase', 'BiocGenerics', 'car', 'colorspace',
  'corpcor', 'degreenet', 'devtools', 'dichromat', 'digest', 'distr',
  'distrEx', 'effects', 'FAiR', 'flexmix', 'flowCore', 'flowViz', 'getopt',
  'ggplot2', 'gpclib', 'graph', 'gRbase', 'hexbin', 'HH', 'igraph', 'igraph0',
  'iplots', 'kernlab', 'ks', 'latentnet', 'lavaan', 'lawstat', 'locfit',
  'maptools', 'memisc', 'mice', 'network', 'optparse', 'party', 'pcaPP',
  'prodlim', 'psych', 'qgraph', 'quantreg', 'RandomFields', 'RBGL', 'Rcmdr',
  'RCurl', 'relevent', 'rgenoud', 'rgl', 'Rgraphviz', 'rJava', 'rms', 'rpanel',
  'segmented', 'sem', 'shapes', 'slam', 'sna', 'snow', 'sp', 'SparseM',
  'spatstat', 'splancs', 'startupmsg', 'survey', 'SweaveListingUtils',
  'TeachingDemos', 'testthat', 'trust', 'XML', 'Zelig', 'cluster', 'foreign',
  'KernSmooth', 'lattice', 'Matrix', 'nnet', 'rpart'
>

I said "n". Maybe you should say yes.

That script tries to do a lot of things, including a global update of all R packages, which you may or may not want. It asks if you want that, say "y" and it will do its thing. I would NOT do that on a "production" computer, I can't trust somebody else's script to fiddle around with my installation. I'd want to understand all of the details. Maybe on your laptop, it OK to say yes.

I asked around about Bioconductor installs and tried to find out why they do it that way. It is simply for user convenience, there is no evil purpose.

If you want to understand how package repositories can be used, and libraries of R packages can be managed, then you don't have to rely on that script. On our server systems, I don't use that script.

To make this work, you need 2 settings. First, you need to decide what directory in your PC will hold the installed package. That is called the "library". To see the directories where you currently have libraries, run this command. I'll show my output:

> .libPaths()
[1] "/home/pauljohn/R/x86_64-pc-linux-gnu-library/2.15"
[2] "/usr/local/lib/R/site-library"
[3] "/usr/lib/R/site-library"
[4] "/usr/lib/R/library"

If I want to throw Bioconductor packages into one of those libraries, all is well.

Read the help page on R's install.packages() function. Note it has arguments for lib and repo.
You explicitly tell R where to get packages and where to put them.

> install.packages("graph", lib="/some/path/you/want", repos="http://some.valid.package.server/whatever")

If you put "/some/path/you/want" that is different from your .libPaths() output, then you need to think about how to use the packages that you installed in a place that is not in your path. That can be done by changing settings in the R startup environment. Or by editing the variable in .libPaths(). Either way, we can work it out.

But on your own computer, don't bother setting lib, just leave out that argument, R will try to install in item [1] in your .libPaths()

On our cluster, I have Bioconductor stuff installed to a separate directory, but here I might be lazy and just use .libPaths()[1], in my home directory.

On the cluster, here's the R script I run to get their packages, which drops
the results into "/tools/lib64/R/bioconductor". As you can see, when I ran that,
the correct version of the BIOCREPO variable was versioned at 2.10. I think now
the correct Bioconductor version is 2.11. Note Bioconductor versions DO NOT
mirror R versions. I'm in R 2.15, but the Bioc version is 2.10 below, and I should
raise that to 2.11.

BIOCDIR < - "/tools/lib64/R/bioconductor"
BIOCREPO <- "http://www.bioconductor.org/packages/2.10/bioc"

install.packages("BiocInstaller", repos = BIOCREPO, lib = BIOCDIR)

install.packages("limma", repos = BIOCREPO, lib = BIOCDIR)


desiredPackages <- c("graph", "ddgraph", "cellGrowth", "DirichletMultinomial")

install.packages( desiredPackages, repos = BIOCREPO, lib = BIOCDIR, dep = TRUE)

install.packages("RBGL", repos = BIOCREPO, lib = BIOCDIR, dep = TRUE)

install.packages(c("biocGraph", "DEGraph","Rgraphviz"),  repos = BIOCREPO, lib = BIOCDIR, dep = TRUE)
Posted in R | Tagged , | Comments Off on R Bioconductor Install (or, where’s my “graph” package?)

Linux create ssh keys

Posted on 2013/03/15 by pauljohn

These instructions are for a Linux system, but if you install Git for Windows ( msysgit.github.com), you will receive a shell and terminal capability.

Usual ssh-keygen will write to ~/.ssh/id_rsa (creates 2 files). In Linux, ~ means $HOME, usually something like /home/your-name-here. In Windows, in the Git BASH terminal, you run "env" and you'll see how it guessed for $HOME.

We recommend this to beef this up

# ssh-keygen -t rsa -b 4096 -f ~/.ssh/my_key_name -C "your-email-address@wherever"

You need the -C thing sometimes because some servers will insist you use a SSH key that is linked to your login. For example, GitHub and Debian Mentors do that.

I did not put put the -C option in this example, by mistake.

Run that,you see this:

$ ssh-keygen -t rsa -b 4096 -f ~/.ssh/my_key_name
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/pauljohn/.ssh/my_key_name.
Your public key has been saved in /home/pauljohn/.ssh/my_key_name.pub.
The key fingerprint is:
b3:b5:59:69:33:17:a1:23:6e:80:e7:d0:48:44:9b:96 pauljohn@hpc

The reason to add -f argument is to name the file so as to protect your old id_rsa in case you rely on it.

Posted in Linux | Comments Off on Linux create ssh keys

RedHat Security Tightening; Tale of One System

Posted on 2013/02/19 by pauljohn

The default RedHat install is possibly more secure than some Linux distributions, but it is also not as tight as we might need.

On one KU RedHat system that I did not fine tune--so the default setup was in place--we saw an effort to attack yesterday. Somebody was running scripts to randomly guess names, passwords, and services.

No security breach occurred, but the massive number of attempts got the domain login server pissed off and now it seems as if nobody can authenticate against the home domain from the one particular machine that was targeted. Well, I'm not sure that's the reason why all our users are unrecognized by the login server now, but the correlation is pretty clear. We were able to log in yesterday, all of these ssh attacks occurred, and now we are blocked from authentication.

While I wait to hear how to fix that, let me illustrate the problem and how I'm fixing it.

The /var/log/secure was full of these, 1000s of them:


Feb 19 10:31:49 CRMDA-009 sshd[17959]: Invalid user jaqueline from 200.222.91.18
Feb 19 10:31:49 CRMDA-009 sshd[17960]: input_userauth_request: invalid user jaqueline
Feb 19 10:31:49 CRMDA-009 sshd[17959]: pam_unix(sshd:auth): check pass; user unknown
Feb 19 10:31:49 CRMDA-009 sshd[17959]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=200.222.91.18
Feb 19 10:31:49 CRMDA-009 sshd[17959]: pam_krb5[17959]: error resolving user name 'jaqueline' to uid/gid pair
Feb 19 10:31:49 CRMDA-009 sshd[17959]: pam_krb5[17959]: error getting information about 'jaqueline'
Feb 19 10:31:51 CRMDA-009 sshd[17959]: Failed password for invalid user jaqueline from 200.222.91.18 port 38534 ssh2
Feb 19 10:31:52 CRMDA-009 sshd[17960]: Received disconnect from 200.222.91.18: 11: Bye Bye
Feb 19 10:31:54 CRMDA-009 sshd[17961]: Address 200.222.91.18 maps to mailserver.abolicao.com.br, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Feb 19 10:31:54 CRMDA-009 sshd[17961]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=200.222.91.18  user=root
Feb 19 10:31:54 CRMDA-009 sshd[17961]: pam_krb5[17961]: authentication fails for 'root' (root@HOME.KU.EDU): User not known to the underlying authentication module (Client not found in Kerberos database)
Feb 19 10:31:56 CRMDA-009 sshd[17961]: Failed password for root from 200.222.91.18 port 38798 ssh2
Feb 19 10:31:56 CRMDA-009 sshd[17962]: Received disconnect from 200.222.91.18: 11: Bye Bye
Feb 19 10:31:58 CRMDA-009 sshd[17963]: Address 200.222.91.18 maps to mailserver.abolicao.com.br, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Feb 19 10:31:58 CRMDA-009 sshd[17963]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=200.222.91.18  user=root
Feb 19 10:31:58 CRMDA-009 sshd[17963]: pam_krb5[17963]: authentication fails for 'root' (root@HOME.KU.EDU): User not known to the underlying authentication module (Client not found in Kerberos database)
Feb 19 10:32:00 CRMDA-009 sshd[17963]: Failed password for root from 200.222.91.18 port 39078 ssh2
Feb 19 10:32:00 CRMDA-009 sshd[17964]: Received disconnect from 200.222.91.18: 11: Bye Bye
Feb 19 10:32:02 CRMDA-009 sshd[17965]: Address 200.222.91.18 maps to mailserver.abolicao.com.br, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Feb 19 10:32:02 CRMDA-009 sshd[17965]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=200.222.91.18  user=root
Feb 19 10:32:02 CRMDA-009 sshd[17965]: pam_krb5[17965]: authentication fails for 'root' (root@HOME.KU.EDU): User not known to the underlying authentication module (Client not found in Kerberos database)
Feb 19 10:32:04 CRMDA-009 sshd[17965]: Failed password for root from 200.222.91.18 port 39309 ssh2
Feb 19 10:32:04 CRMDA-009 sshd[17966]: Received disconnect from 200.222.91.18: 11: Bye Bye
Feb 19 10:32:06 CRMDA-009 sshd[17967]: Address 200.222.91.18 maps to mailserver.abolicao.com.br, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Feb 19 10:32:06 CRMDA-009 sshd[17967]: Invalid user src from 200.222.91.18
Feb 19 10:32:06 CRMDA-009 sshd[17968]: input_userauth_request: invalid user src
Feb 19 10:32:06 CRMDA-009 sshd[17967]: pam_unix(sshd:auth): check pass; user unknown
Feb 19 10:32:06 CRMDA-009 sshd[17967]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=200.222.91.18
Feb 19 10:32:06 CRMDA-009 sshd[17967]: pam_krb5[17967]: error resolving user name 'src' to uid/gid pair
Feb 19 10:32:06 CRMDA-009 sshd[17967]: pam_krb5[17967]: error getting information about 'src'
Feb 19 10:32:08 CRMDA-009 sshd[17967]: Failed password for invalid user src from 200.222.91.18 port 39541 ssh2
Feb 19 10:32:09 CRMDA-009 sshd[17968]: Received disconnect from 200.222.91.18: 11: Bye Bye
Feb 19 10:32:10 CRMDA-009 sshd[17969]: Address 200.222.91.18 maps to mailserver.abolicao.com.br, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!

After that problem started, then users could not log in anymore, the domain rejects the requests thusly (from /var/log/secure):

Feb 19 11:40:06 CRMDA-009 pam: gdm-password[18264]: pam_unix(gdm-password:auth): check pass; user unknown
Feb 19 11:40:06 CRMDA-009 pam: gdm-password[18264]: pam_unix(gdm-password:auth): authentication failure; logname= uid=0 euid=0 tty=:0 ruser= rhost=
Feb 19 11:40:06 CRMDA-009 pam: gdm-password[18264]: pam_krb5[18264]: error resolving user name 'pauljohn' to uid/gid pair
Feb 19 11:40:06 CRMDA-009 pam: gdm-password[18264]: pam_krb5[18264]: error getting information about 'pauljohn'
Feb 19 11:40:06 CRMDA-009 pam: gdm-password[18264]: gkr-pam: error looking up user information for: pauljohn

How I'm fixing this on RedHat.

1. Install the RPM denyhosts, edit the config to make at least one change. in /etc/denyhosts.cfg, change the file where it lists banned systems. The reason for this is that we don't want the denyhosts program to obliterate our existing hosts.deny file, we want it to create its own enemy list and then we take not of it in hosts.allow.

# Most operating systems:
HOSTS_DENY = /etc/denyhosts.blocked

2. Edit /etc/hosts.deny, insert 1 line

ALL:ALL

That means we are blocking access to all services on all ports from all places, by default.

3. Edit /etc/hosts.allow, insert lines to make it so that we ONLY offer ssh service on this system, and we ONLY offer it to a trusted domain. We want to allow in users who have IP numbers such as 10.222.xxx.xxx, for example. In this example code,

portmap: 10.222.
ALL: 127.0.0.1
sshd : /etc/denyhosts.blocked : deny

sshd: 10.222.
sshdfwd-X11: 10.222.

If users from more IP ranges have to be let in, OK. Add them.

Even users who are within the valid range can be blocked by denyhosts.

Posted in Linux | Comments Off on RedHat Security Tightening; Tale of One System

Emacs settings for Windows and the Rterm problem

Posted on 2013/02/18 by pauljohn

Good page:

http://cbio.ensmp.fr/~thocking/primer.html

Shows how to use .Rprofile and .emacs file. This is the old fashioned way,

(setq inferior-R-program-name "/path/to/R");;

I need to return and get this right, once and for all. So I started this page as a reminder to self.  Google for ".emacs file ess-directory-containing-R".  that is the way to set it now.

Posted in Uncategorized | Comments Off on Emacs settings for Windows and the Rterm problem

Installing LyX 2.0.5 in Windows, step by step

Posted on 2013/02/11 by pauljohn

## Paul E. Johnson <pauljohn at ku.edu>
## 2013-02-09
## Step by step LyX install Notes

## Test operating system is Windows-7 32 bit virtual machine.
## User making install is NOT a "limited" account. (Recall that
## the limited account is not the default account type, but it is the
## type that all Windows
## users should use as their primary day-to-day login).
## I expect this install would fail if run by a limited user.

## I'm writing down every step.

## Go to http://www.lyx.org

Download

LyX-2.0.5.1-Bundle-4.exe

Right click, run as administrator.

Say yes to proceed
License prompt: agree
Install for anyone using this computer

Destination Folder. I get rid of all spaces where possible

C:\Program Files\LyX-2.0

Select Components.
Leave LyX, File associations, Desktop icon
Check JabRef
I left dictionaries.
Thesaurus is new feature to me, never tried before, will try now.
--Check Thesaurus, click the + and UNCHECK lots of languages.

Name Start Menu Folder whatever you like (I left default)

Now you see the benefit of the "Bundle" install. The download included
basic-miktex, jabref (bibtex editor). Almost immediately, menu says:

LyX Setup:
Now the installer of the LaTeX-distribution "MikTeX" will be launched.
To install the program, press the "Next" button in the installer windows
until the installation begins.
!!!Please use all default optinos of the MikTeX installer!!!

That emphatic statement there (I'm almost sure) relates to the problem
that the MikTeX distribution will need many addon packages and there
has been trouble with the packages that download "on the fly" or without
user permission. So I'm following their guide to see if their way works.
In the past, there was trouble.

I accepted the MikTeX conditions, it asks where to Install MikTeX.
I'm going to get rid of the space:

C:\Program Files\MiKTeX

Now the problem. LyX instructions said leave defaults, but I cannot
believe that means I have to accept A4 paper. So I refuse.

But the option "Install missing packages on the fly" is marked "Ask me first"
and that's default, but it has been a big big trouble for me in the past.
The problem was that a flaw in Windows or MikTeX or LyX caused the popup
permission window to fail, and so LyX would hang. I'll accept the
default now, and then see what happens.

Now the MiKTeX install window appears and it is doing a lot of work.
This part takes at least 10 minutes. At several points, I thought it  was
hung. I'm doing this in a place where the firewall allows http traffic,
but it might block other download types that the system tries. We will
see.

While I wait, I'd mention that I've wrestled with the LyX install in
Windows many times and on our remote desktop servers, we experienced
the same problem.  Every time a new user tries to use LyX for the first time, there
is a super long delay, as long as 40 minutes, before LyX works.
Sometimes it turns out that some prompt window has failed to open.
Sometimes, we find that the LaTeX distribution is trying to download more packages.
Eventually it does work, but most of us give up, concluding LyX does
not work. The system administrators found a work around, I posted it
in the Lyx-devel email list.

I am pretty sure the hold up is just Windows and MiKTeX not working
correctly together for users who are not administrators.  To work with
the MiKTeX stuff, one really needs admin power--since it is changing
stuff in C:\Program Files\MiKTeX.  But the LyX users are not administrators,
so when LyX needs a package, it can't get it.

One solution would be to install all MiKTeX packages with the MiKTeX
installer.

Anyway, this is a long standing problem which can be solved with patience.
On a user's laptop, where there is only one user to worry about, we have
always found a way to get it going.
Usually, after the initial 30 minute wait to view the very first
document, then LyX will be fine.

As the MiXTeX install is proceeding, I notice again this weird/frustrating
thing that Windows does. It pops up windows below other windows.

I thought the install was working, but actually it was waiting for me to
say "NEXT" in a menu that had popped up BELOW the Setup popup window.

I realize there are actually two windows on screen, one on top of the other.

1. LyX 2.0.5.1 Setup (seems to indicate it is working)

2. Basic MiKTeX 2.9.4757 Installer (32-bit)

The Basic MikTex was underneath the other, waiting for me to say next.

So, when you think the install is dead, move the windows around and hunt for pop up windows.

After I hit "next", then the install proceeds.

A new popup appears:

LyX 2.0.5.1 Setup

The LaTex distribution "MikTeX" will be used together with LyX. It is recommended to
install available MiKTeX updates using the program "MikTeX Update Wizard" before you
use LyX for the first time. Would you now check for MiKTeX Updates?

I said yes.

A window pops up asking where to check for updates. I Said find the nearest repository,
and there was a long delay while "Searchin..."

And then The MikTeX install CRASHED.

Popup window says:

MiKTeX Update Wizard has stopped working.
The Details indicate:

Problem Signature
Problem Event name APPCRASH
Application Name     miktex-update_admin.exe
Application Version:     2.9.4745.0
Application Timestamp:  50dd63c1
OS Version   6.1.7600.2.0.0.256.48

I've seen APPCRASH from LyX before, it is one of the Windows problems. I've
never gotten a clear explanation, nobody I know who uses Windows cares enough
to learn what's wrong with it or why that happens. Hopefully, some Windows
user reads this and tells me what is wrong with it. Over the years, I've
come to conclude APPCRASH is just another symptom of the Windows OS problem.
The OS if flaky enough so that only the Microsoft company can write good software
for it, and the Microsoft company is not interested in fixing that because it
perpetuates their monopoly on Office software.  Recall, MS did loose the
Netscape & Word Perfect lawsuits, only to be excused by the US Government.

Now the LyX 2.0.5.1 Setup window is back on top. It says:

Configuring LyX (MiKTeX may download missing packages, this can take some time)...

I don't know if the APPCRASH will frustrate this. I'm waiting. The encouraging
thing is that the green progress bar in the panel is almost all the way across.

I'll click the "Show details" button to try to tell if it is actually going anything.

Good, it shows lots of downloads going on.

downloading from http://ftp.fernuni-hagen.de/ftp-dir/pub/mirrors/www.ctan.org/systems...
extracting files from ttk.tar.lzma ...

As long as that's doing something, we are OK

Wow, this is slow. It's been 15 minutes.

And, as I watch the downloads, I'm remembering the problem that will come up in the future when more packages are needed.
If I run LyX as a non-administrator, will the packages be installed?  Will LyX be able to trigger MikTeX to ask me for permission and a password? In 2011, the answer was NO, but I always have hope. But the APPCRASH in the MikTeX updater makes me doubtful.

Hooray. It finished.

The final panel asked me if I want to start LyX.

Nothing appears to happen.

Did LyX die? Is it doing that super-slow-startup problem I've seen before?

I'll try Alt-Ctrl-Delete to get task manager open. If LyX is running, it should show.
I see nothing, so I'll try a double-click on the LyX icon

I see the little circle for 20 seconds, then LyX opened with a welcome window.
And a nice introduction to LyX it is.

I will try the View/PDF pull down to see what happens.

GREAT.  Right away, the PDF opens on the screen. It is using the PDF viewer from the
TeXWorks project, which is part of MiKTeX.  In the old days, it used to use a
program called Yapp. I don't mind.

If you try the same exercise, you will note the fonts in that "splash.lyx" document
LOOK HORRIBLE. They are all fuzzy.  The default fonts are for postscript, not
PDF. This is one of the historical frustrations with LaTeX in general and LyX in
particular. Every single new user who comes along says "my PDF output looks
terrible" and the experts respond, "Oh, did you want good looking output? Go
read a 92 page manual about postscript, Type I, Type II, True Type, and some
other font crap and check back later." It drives me crazy.

Here's the instant fix. In the LyX document, open Document -> Settings -> Fonts
and change the font type to "Latin Modern". You have to do this for
Serif
Non-Serif
Typewriter
separately. If you don't like LatinModern, Palatino works well too. And, in fact,
the True Type fonts are usually workable too.

The multiplicity of font formats, their non-interchangeability is a long standing
problem that people smarter than we are are fixing as we speak.

I'm wondering if the TeXWorks PDF viewer is not so great. I'm going to open LyX
Tools -> Preferences to select a viewer, either Adobe Acroread or one of the GTK
viewers (Okular, etc).

That's the end of session 1, more will follow

##### Session 2

One goal is to make LyX work with R and Sweave.

The LyX team has worked on this, now LyX has an Sweave
module that may make this easier for beginners. I will
try this out "barefoot," as if I did not plan to make a lot
of customizations.

I open LyX, and then Document -> Settings -> Modules
and choose Sweave.

Note in the small pnanel it says it requires sweave, fancyvrb.

The Sweave.sty file is distributed along with R.

I went into the MiXTeX install, to this spot:

C:\Program Files\MikTeX\tex\latex

and In there I created a folder "R".

Then I copy the Sweave.sty file from the R distribution into
that folder.  The Sweave.sty file for R file is under

C:\Program Files\R\share\texmf\tex\latex

Copy that into the MikTeX location.

BUT WAIT!  Its not ready yet.

Even if that were the only file that needed insertion, we still
neeed to make MikTeX aware of it. (On unix/linux, we'd run 'texhash'
in a terminal to force a re-scan.

In Windows, Click the Start Menu -> MixTeX -> Admin -> Package Manger.

Recall the LyX panel warned it needed Sweave and also fancyvrb.
My MiKTeX seems to say that fancyvrb is already installed. So all
I need to do is force a re-scan of the style directories. How to
do that in Windows?

In MikTeX Package manager, click "Repository" -> "Synchronize".

While you are in there, you can install any other latex packages
you might need.

You could also do "Task" -> "Update Wizard". For me that dies, says
"The update wizard could not be found". Possibly related to the
APPCRASH i had before.

Now, lets see if this will work in LyX. Want to try it out?

STOP. You have to let LyX notice the MikTeX changes.

In Lyx, Click "Tools" -> "Reconfigure"

Then close LyX.

Restart LyX. Start a new document. Don't worry about details
in Document -> Settings yet, we just need to go step by step.

Type something, try to compile it (View -> PDF).

All good?  Now go into Document -> Settings -> Modules and
activate the Sweave module.

Hm. Drag. Activation still fails, saying

The module sweave requires a package that is not available
in your LaTeX installation, or a converter that you have
not installed. LaTeX output may not  be possible. Missing
prerequisites:   sweave->latex. See section 3.1.2.3
of the User's Guide for more information.

I've been here before. We need to figure how LyX is supposed to call R.

About 10 years ago, I started the documentation for this in the
LyX website, I notice people have been updating. Yeah. Go see:

http://wiki.lyx.org/LyX/LyxWithRThroughSweave

In the PATH prefix bar, add semicolon and

C:\Program Files\R\bin

(that's where my R is)

Now make LyX rescan by Tools -> Reconfigure, then close LyX

Restart LyX, File -> New, then Document -> Settings -> Module
choose Sweave, add, and there are no more errors.

Now go back to the document and type anything and make sure it compiles.

Don't try any Sweave stuff yet, just type text or such.

Now we test out Sweave MY WAY. I've been doing this a long time, and I DO NOT like the effort to make the R code blend "seamlessly" into the on screen document. I want the R code to be AS VISUALLY distinct as possible. So I put my R chunks in TeX boxes.

Hit the TeX button and a red box appears, and type this

<<>>=
 x <- rnorm(5)
 mean(x)
 var(x)
@

(Correction 2015-10-13: don't forget these begin with <<>>= and end with @. Arguments like the chunk name, figure parameters, etc can be inside the <<>>.

View -> PDF, and for me, it worked?

If it did not work for you, then review the steps and find out why.

If it did work, then your next step is to learn some of the
arguments that you can put between the <<>>= in the R chunk.

The whole Sweave idea was created by Friedrich Leisch and he has
some good documents about it.

I've started a web page about latex and sweave tips here:

https://pj.freefaculty.org/latex

That page points at to some Sweave starter documents.

Over the long run, I think this is where you want to be.

1. Don't use the default Sweave style. It is based on the
LaTeX Verbatim class, which is bad in many ways. Instead, switch
to use Frank Harrell's Sweavel.sty class, which uses LaTeX
listings instead of Verbatim to render the R code and output.
It is much more versatile and beautiful.  So you need to copy
the Sweavel.sty file from his website, put it in the same
place where you put the R Sweave.sty file, make MikTeX reconfigure.

Then you put some magic in the LyX document preamble (Document->
Settings -> Preamble) and make a few boiler plate changes in the
document. I've used this in at least 50 documents in the last
5 years and it can be made to work out.  Just about everything
under

https://pj.freefaculty.org/guides

uses that.

2. Set the various R options to put the R output into a
separate folder, and also to BREAK UP the LaTeX output from
Sweaving into separate files. In all my chunks after that,
I start like so

<<chunk10, ..other-options-you-use, include=F>>=
R code here
@

That causes the R output to be named "prefix-chunk10" in the
R output folder.  If the code makes a figure, it has a figure
"prefix-chunk10.pdf".  So in LyX document, I insert that
tex output and the figure exactly where I want. In other workds,
I no longer let Sweave decide where to put the output. I put
it in myself

\input{file-name-here}

\includegraphics[]{file-name-here}

The advantage of this setup is that we can "TURN OFF" the Sweave module, and the document can still be compiled because the output from the previous Sweave is saved in a folder.

I don't want to write out all the details here, I only mention
it because, if you go looking into my guides folder, you see
I almost always have a sub-folder "plots" and that is the place
where I have R dump the figures and the input and output code.
That stuff is there, waiting to be inserted into the document however
I like.

3. To make a Beamer presentation using Sweave, a whole horrible
layer of awfulness comes into play.  Both the Sweave approaches,
either the default Sweave.sty (which uses LaTeX Verbatim) and
Sweavel.sty (which uses Latex listings) will not cooperate with
the default Beamer slide. It is necessary to start a slide in
TeX code like so

\begin{frame}[containsverbatim]
\frametitle{your title here}

type the slide in here

\end{frame}

It has been a long standing wish for LyX to more gracefully
interact with customized frames. But until it is completely
solved, I'm typing all the frames in TeX boxes.  There is
talk in the lyx-devel list about making frames that allow a
[fragile] marker to achieve same purpose, but that doesn't really
get the job done because it isn't exactly the same as
[containsverbatim] and it also does not allow the other
frame option that I often use. That is [allowframebreaks].

By far the most horrible hassle in writing slides is making
sure that the content fits on a single screen.  There's no
way to tell if it fits without compiling it. And while you
are in the creative mood, you don't want to stop writing
to re-arrange content so it fits on a screen.  In that case,
the easiest thing to do is to open a multi-screen frame like
so

\begin{frame}[containsverbatim, allowframebreaks]
\frametitle{My Frame is }

write whetever you want here.

\end{frame}

The default output is a little ugly, it uses some roman numerals in the frame names in the output. But that is easily fixed by a customization of the document prefix. Most of my stuff in the guides folder incorporates that.

Posted in LaTeX/Lyx, Uncategorized, Windoze | Comments Off on Installing LyX 2.0.5 in Windows, step by step

Debian multiarch: Unexpected complication when building amd64 packages: Can’t install!

Posted on 2013/01/23 by pauljohn

This is a multiarch issue I had not considered before. Have you seen it? I never wanted to be a "cross compiler", I really only want to build amd64. But I have some i386 libraries for a particular program (acroread).

I've just learned that, if I build amd64 packages, I can't install them for testing because I've not also built the i386 packages. I built the packages, watch their install 1) proceed, 2) fail to finish (leaving system in very inconsistent state)

pauljohn@pols-124:cairo-1.12.10$ sudo dpkg -i libcairo2_1.12.10-1_amd64.deb (Reading database ... 311111 files and directories currently installed.)
Preparing to replace libcairo2:amd64 1.12.10-1 (using libcairo2_1.12.10-1_amd64.deb) ...
Unpacking replacement libcairo2:amd64 ...
dpkg: error processing libcairo2:amd64 (--install):
package libcairo2:amd64 1.12.10-1 cannot be configured because libcairo2:i386 is at a different version (1.12.2-2)
Errors were encountered while processing:
libcairo2:amd64

That's really inconvenient! I don't understand why there has to be a linkage between the shared library versions on amd64 and i386. Aren't they separate?

You ask "why cairo?" I am curious to know if a new libcairo2 fixes a little bug in Evince (invisible vertical quotes: https://bugs.freedesktop.org/show_bug.cgi?id=59636#c4). So I worked through the packaging for cairo-1.12.10. dpkg-buildpackage -rfakeroot gives me the goods (after fiddling some patch fuzz):

cairo-perf-utils_1.12.10-1_amd64.deb
libcairo2_1.12.10-1_amd64.deb
libcairo2-dbg_1.12.10-1_amd64.deb
libcairo2-dev_1.12.10-1_amd64.deb
libcairo2-doc_1.12.10-1_all.deb
libcairo2-udeb_1.12.10-1_amd64.udeb
libcairo-gobject2_1.12.10-1_amd64.deb
libcairo-script-interpreter2_1.12.10-1_amd64.deb

I expect your answer will be "yes, it really is that hard, you have to learn how to compile for i386 too". I'm trying (http://wiki.debian.org/Multiarch/HOWTO), but not making progress. I'm like a collection of monkies trying to type the Bible at random, I'm afraid.

pauljohn@pols-124:~$ sudo apt-get build-dep -a i386 libcairo2
[sudo] password for pauljohn:
Reading package lists... Done
Building dependency tree
Reading state information... Done
Picking 'cairo' as source package instead of 'libcairo2'
The following packages have unmet dependencies:
libfontconfig1-dev:i386 : Depends: libfreetype6-dev:i386 (>= 2.1.7) but it is not going to be installed
E: Build-dependencies for libcairo2 could not be satisfied.

pauljohn@pols-124:~$ sudo apt-get install libfontconfig1-dev:i386
Reading package lists... Done
Building dependency tree
Reading state information... Done
You might want to run 'apt-get -f install' to correct these:
The following packages have unmet dependencies:
libcairo2 : Breaks: libcairo2:i386 (!= 1.12.10-1) but 1.12.2-2 is to be installed
libcairo2:i386 : Breaks: libcairo2 (!= 1.12.2-2) but 1.12.10-1 is to be installed
libfontconfig1-dev : Conflicts: libfontconfig1-dev:i386 but 2.9.0-7.1 is to be installed
libfontconfig1-dev:i386 : Depends: libexpat1-dev:i386 but it is not going to be installed
Depends: libfreetype6-dev:i386 (>= 2.1.7) but it is not going to be installed
Depends: pkg-config:i386 but it is not going to be installed
Conflicts: libfontconfig1-dev but 2.9.0-7.1 is to be installed
E: Unmet dependencies. Try 'apt-get -f inst

And I stop there, seeing that the failed install of the amd64 packages has now broken everything. I need to back-track to find a way to fix that and then hide in the safety of Wheezy.

Posted in Linux | Tagged , , | Comments Off on Debian multiarch: Unexpected complication when building amd64 packages: Can’t install!

Big LyX Sweave output breakthrough

Posted on 2013/01/18 by pauljohn

I've tried to set up LyX converters and copiers many times to achieve a particular result. Today I've succeeded for the very first time.

From a single export, I want

  1. pdf output for a lecture
  2. article format output for the same lecture (for people who want to save paper)
  3. The R code that goes with the lecture

I need to have all 3 synchronized, ALL the time. I mirror my work directories to the Web pages, and it is not great when they go out of sync. I've got to do this several times a week, and it is a hassle if I have to remember lots of steps.

Here's the example output

first-R-03.pdf

first-R-03-article.pdf

I suppose you can just go look for  output, I'm testing this on my new elementary R exercises https://pj.freefaculty.org/guides/Rcourse/First-R/

And the R code file that goes with is called first-R-03.R.

In case you have never used Sweave, or LyX, some little explanation is in order. The internal conversion in LyX goes like this. The document I'm editing in LyX is really a noweb "literate" programming file, otherwise known as an Sweave document. That is words and comments with bits of R code chunks buried throughout.  If I were editing raw LaTeX, it would be an Rnw file. LyX dumps out its content and converts it into a proper Rnw file as the first step in the document processing system. That Rnw is converted to LaTeX by R's Sweave command. That sends the R code chunks to R for processing, and output files for the R stuff are dumped out, along with a LaTeX file.  I prefer the command line "R CMD Sweave filename.Rnw". The sweaving produces a tex file, which is then converted to pdf by the usual LaTeX methods.

Students may want to know what R produced the figures in my document, and I need that to be immediately available. So, in addition to generating the pdf output, I also need to extract the R commands from the Rnw file. That is called "tangling" in the R parlance. I would have called it "untangling", since it extracts the R commands that are embedded in the Rnw file. But I didn't invent the whole process, I'm just a fan of it. Thanks to Professor F. Leisch for thinking of the whole idea, working it out beautifully, and documenting it excellently.

There's a wrinkle. My Rnw file has the option to split the output of each command into separate files in the plot directory.  (look in the web site, you'll see a "plots" folder. That makes the output much easier to work with and integrate into documents in a "fine grained" user controlled way. I've got separate little output files, I can input them wherever I want.

However, that approach does not generate a single R file when the tangling is done. Joy of joys, however. I can use perl to change the split option in the Rnw file AFTER it has generated its output, and then I tangle that file after changing the split setting to false. Genius. I'm sorry to brag, but this has been a pain in the rump for about 3 years.

How to achieve this long sought after magic?  The magic bean is the abuse & misuse of LyX's copier customization. The usual document export would create the first pdf and copy back to the document directory. A copier script does that.

Today I realized that I can hack a new copier script that completes that first part, returns the pdf output to my document directory,  but then also does the other bits of work as a part of the same motion.  The script looks at the output target directory, which is my original document directory.  It follows the pdf output file back there, it scans the files, and then processes them to create a new article format document.  I can't say this is the most perfect idea ever, but it is the first way I've found that appears to work in a practical way.

How? fiddle together some scripts, integrate them into LyX preferences.  I've never had much luck using the LyX graphical preference editor, but I do succeed while editing the preference file in the lyx configuration directory.

About the Sweaving: I've been doing it this way for a decade, before LyX integrated its own approach to Sweave. I prefer my approach for various reasons. My "pjconvert-small.sh" is doing essentially the same work as the LyX build in sweave->pdf translation.  The built in thing in LyX does  not have the automatic multi-generation of outputs that I'm aiming for here. And it leaves some trash files like Rplots.pdf in my document directory. But otherwise, it works in exactly the same way.

I'm afraid I'll forget the details, so here they are. All the scripts I write have pj at the front because I need to keep them separated from the scripts that other people that are piling up. And I'm egotistical.

If, in the future, I have to cave in and let the built in LyX converter do its work, the copier can still be used with that so as to generate the R and article output after the main pdf is created. Yay. I'm so happy.

In the ~/.lyx/preferences, put some new document types, converters, and classes.

<pre>

# FORMATS SECTION ##########################

\format "fen" "fen" "FEN" "" "auto" "auto" ""

\format "agr" "agr" "Grace" "" "auto" "auto" "vector"

\format "pdflatex6" "tex" "LaTeX (pdflatex 6)" "" "" "%%" "document,menu=export"

\format "pdflatex7" "tex" "LaTeX (pdflatex 7)" "" "" "%%" "document,menu=export"

\format "noteedit" "not" "Noteedit" "" "auto" "auto" "vector"

\format "pdf7" "pdf" "PDF (Beam + Article)" "" "evince" "" "document,vector,menu=export"

\format "pdf6" "pdf" "PDF (Beamer)" "" "evince" "" "document,vector,menu=export"

\format "pdf2" "pdf" "PDF (pdflatex)" "F" "evince" "" "document,vector,menu=export"

\format "r" "R" "R/S code" "" "" "" "document"

\format "tgif" "obj" "Tgif" "" "auto" "auto" "vector"

# CONVERTERS SECTION ##########################

#

\converter "sweave" "pdflatex7" "pjconverter.sh $$p$$i $$p $$r $$b" ""

\converter "sweave" "pdflatex6" "pjconverter-small.sh $$p$$i $$p $$r $$b" ""

\converter "pdflatex7" "pdf7" "pdflatex $$i" "latex=pdflatex"

\converter "pdflatex6" "pdf6" "pdflatex $$i" "latex=pdflatex"

#
# COPIERS SECTION ##########################
#

\copier pdf7 "pjBeamerMultiOutCopier.sh $$i $$o"

##############################.lyx/preferences################

</pre>

And the  required scripts

 

pjconverter.sh

<pre>

#!/bin/bash

## Arguments. $1 is Rnw name

## $2 working directory

## $3 input directory

## $4 input file without extension

 

rm -rf $4.pdf

rm -rf "$2/plots"

rm -rf "$2/*.pdf"

R CMD Sweave $1

if [ -d $2/plots* ]; then

rsync -rav $2/plots* $3;

fi

pjrtangle.sh "$1" "$3"

sleep 1

exit 0

</pre>

pjconverter-small.sh

</pre>

#!/bin/bash

## ONLY creates the pdf file.
## Arguments. $1 is Rnw name
## $2 working directory
## $3 input directory
## $4 input file without extension

rm -rf $4.pdf
rm -rf "$2/plots"
rm -rf "$2/*.pdf"
R CMD Sweave $1
sleep 1
exit 0

</pre>

pjBeamerMultiOutCopier.sh

<pre>

#!/bin/bash
FROMFILE=$1
TOFILE=$2

##First, copy the pdf output that was already made back to document directory
echo "Now try cp $FROMFILE $TOFILE"
cp $FROMFILE $TOFILE

FILEBASE=`basename $2`
FILENOEXT=${FILEBASE%.*}

WORKDIR=`dirname $FROMFILE`

DOCDIR=`dirname $TOFILE`
cd $DOCDIR

cp "$FILENOEXT.lyx" "$FILENOEXT-article.lyx"

perl -pi.bak -e 's/sweavel-beamer/sweavel-beamer-article/' "$FILENOEXT-article.lyx"
lyx -e pdf6 "$FILENOEXT-article.lyx"

mv "$FILENOEXT-article.lyx" /tmp
mv "$FILENOEXT-article.lyx.bak" /tmp

sleep 1
exit 0
</pre>

 

pjrtangle.sh

<pre>

#!/bin/bash

## take an Rnw that has split=T, change to F
perl -pi.bak -e 's/split=T/split=F/' $1

R CMD Stangle $1

FILE=`basename $1`
FILENOEXT=${FILE%.*}

rsync -av $FILENOEXT*.R $2
exit 0

</pre>

 

Posted in LaTeX/Lyx, R | Comments Off on Big LyX Sweave output breakthrough

Linux: Hiding the “desktop panel” when you want more screen real estate

Posted on 2013/01/14 by pauljohn

It wasn't always the case that the panels and other such devices on the edges of the screen were "always on top" when visible. I hate the current trend of making them "un-cover-upable". It is very inconvenient to the way I work.

In the Window Manager Compiz on Debian Linux, with the XFCE4 desktop, I've found a workable solution. I can overlap the panels when I want, or bring them to the top when i want. I have an almost workable solution, in case somebody else agrees with my view on this.

Here's the idea. In Compiz, use the settings to liberate window placement and put the panels at the bottom of the stack.

In ccsm, "Window Rules", make these 2 changes:

Above: (title=Run Application) & (type=Dialog)
Below: (role=Panel) | (role=DOCK) | (type=xfce4-panel)

In "Move Window" panel, uncheck "Constrain Y".

This *at least* allows windows to go on top of the panel and off the
edge of the screen if desired.

That still leaves the problem that the panel can get buried under a
lot of windows and there seems no way in Compiz--no setting--that
allows us to give the panel focus again so it can rise to the top when
we want it. Its necessary to move all the windows out of the way. I
wish the WindowRules had an option "allow focus click" or something
like that.

I've found a workaround in a program called wmctrl. It can be used to
toggle the behavior of the panel, to make it come to the top or go
back down. Install that, then run this to toggle the panel visibility.

$ wmctrl -r xfce4-panel -b toggle,above

Its really fun to run that over and over 🙂

Bind that command to a key stroke. I chose to map it to the Super-c key, where
Super is the thing with the Windows Menu key.

In the end, I ended up having to put the command in a separate excutable script
file. Digging through my setup, here's what I have in ~/.config/compiz/Defaults.ini. In the

In the section 

[commands]

as_command4 = togglePanelViz.sh
as_run_command4_key = c

The script file in ~/bin/togglePanelViz.sh had this one line:

wmctrl -l | grep panel | awk '{print "wmctrl -i -r" $1 " -b toggle,above"}' | bash

That scans the wmctrl output, grabs all panels, and toggles.

So now it works well to Drag a window over the panel. The part of the panel that is not covered still shows.

I have not solved all of the frustrations. The initial window placement algorithms still respect the panels as special super-protected things, so windows won't automatically move over the panel. However, you can
1. Alt-left-click drag a window over the panel
2. Alt-middle-click to resize a window over the panel

It still is now allowed to "edge drag" and resize over the panel.

Posted in Linux | Tagged , , | Comments Off on Linux: Hiding the “desktop panel” when you want more screen real estate

Lyx: Customized bullets lists and the Simpsons!

Posted on 2013/01/14 by pauljohn

Without further adieu! Working example output:

https://pj.freefaculty.org/scraps/simpsons/simpsons-2.pdf

Lyx file:

https://pj.freefaculty.org/scraps/simpsons/simpsons-2.lyx

Even if you don't want the Simpsons as your bullets in lists, you
might still look this over because it shows how you can put in ANY
valid LaTeX symbol a the bullet for a particular item or for an entire
list. And each list in a presentation can have different bullet
symbols.

I did not find any easy LyX documentation on this. If there is some,
and you wrote it, I'm sorry to overlook you. .

1. Easy, direct way, no LyX module or new package required:

In case you did not know this yet, in LyX, after you create a list
item, and the bullet symbol is showing, you can enter ERT mode and
just put the name of a symbol in brackets, like [\Bart]. That replaces
the default bullet with your symbol. Also things like [$\gamma$] work
if you don't have the Simpson's characters. You can intersperse those
customized bullet symbols with regular bullets. If you go shopping in
the MASSIVE LaTeX symbol collection (see slide 1 in the pdf), you will
probably see symbols you would love in your lists.

2. Use enumitem, gain ability to change bullet for entire list easily:
ERT [label=\Bart]

If you turn on the LyX module to Customize Itemized Lists (enumitem),
then you gain some new powers, such as the ability to declare a label
in the first item of any list and that is used automatically for all
remaining list items.

I could use your help.

Here are problems I did not solve. They are listed in the pdf, but in
case you miss them

1. I can't find a way to re-size the metafont symbols, some (Marge)
are way too big

2. The bullets are aligned with the bottom of the text line, but it
would be more usual to have them aligned with the center of the
multi-line item. Ideas?

3. In Lyx-2.0.3, at least in this example file, as soon as I turn on
the module to Customize lists, then all default bullet symbols are
lost. If I don't specify a label for each list, I get no bullets.
That is a new feature/bug for me, I've not seen it before.

If anybody has ideas about these things, let me know. I may post that
example into the LyX Wiki, if I can reach the conclusion about whether
1 and 2 can be "solved" and if 3 is a problem unique to this document
(probably because I've damaged some default setting somewhere).

Posted in LaTeX/Lyx | Tagged | Comments Off on Lyx: Customized bullets lists and the Simpsons!