The Truth

What do you think the truth is? That is what do you think the concept of “truth” actually means? Is it an absolute- a destination that you can reach if you just try hard enough? Or is it something else? A road that stretches out in front of you and constantly changes as you progress and add more evidence?

Well?

TheTruth_comic

Top Posts of 2013

Although I started blogging in 2012, 2013 has been my first year of blogging. It’s been fun so far if a bit sporadic. I’ve posted approximately once a week, which is a bit less than I’d like. And I’ve had some fun. My top posts for the year are listed in order below. Looking forward to continuing to blog, and improve, in the coming year and beyond. My blog resolution for 2014 is to post more frequently but to also work on a few posts that are more like mini-papers, studies of actual data that’s interesting to the scientific community similar to my analysis of review times in journals. (Caveat: this ranking is based on absolute numbers so short-changes more recent posts that haven’t had as much time to be viewed.  But really I think it’s pretty reasonable)

I had some failures in 2013 too. Some posts that I was sure would knock it out of the park, but didn’t garner much interest. Also, I started a series (parts 1, 2, 3, 4, 5, 6) that was supposed to chronicle my progress on a computational biology project in real time. That series has stalled because it was a bit harder to put together the project than I thought it would be (this is not surprising in the least BTW) and I ran into other more pressing things I needed to do. I’m still planning on finishing this- it seems like a perfect project for the Jan-Feb lull that sometimes occurs.

Top Posts of 2013 for The Mad Scientist Confectioner’s Club

  1. Scientific paper easter eggs: Far and away my most viewed post. A list of funny things that authors have hidden in scientific papers, but also of just funny (intentionally or not) scientific papers. And these keep coming too- so much so that I started a Tumblr to add new ones.
  2. How long is long: Time in review for scientific publications/Time to review for scientific publications revisited: These two posts have analysis I’ve done of the time my papers spent in review. After some Twitter discussions I posted the second one that looked at how long the papers took to get their first review returned, which is more fair to the journals (my first post looked at overall time, including the time that I spent revising the papers). Look for a continuation of this in 2014, hopefully including contribution of data from other people.
  3. Eight red flags in bioinformatics analyses: I’m still working on revising this post into a full paper since I think there’s a lot of good stuff in there. Unfortunately on the back burner right now. However, I did get my first Nature publication (in the form of a Comment) out of the deal. Not bad.
  4. Reviewer 3, I presume?: This post was to recap the (moderate) success of a Tweet I made, and the turning of that Tweet into a sweet T-shirt!
  5. Gaming the system: How to get an astronomical h-index with little scientific impact: One of my favorite posts (though I think I wrote it in 2012) does a bit of impact analysis on a Japanese bioinformatic group that published (and still publishes) a whole bunch of boilerplate papers- and got an h-index close to 50!
  6. How can two be worse than one? Replicates in high-throughput experiments: I’m including this one so that this list isn’t 5 long, and also because I like this post. This is essentially a complaint about the differences between the way that statisticians and data analysts (computational biologists, e.g.) see replicates in high-throughput data and how wet-lab biologists see them. It has yielded one of my new favorite quotes (from myself) that’s not actually in the post: “The only reason to do an experiment with two replicates because you know replicates are important, but you don’t know why.”

Have a great New Year and see everyone in 2014!

I dream of science

I had a dream last night- after yesterday hearing about possible furloughs at the lab due to the government shutdown. Here it is:

I was trying to go into a building and needed to go through security. Now that I think of it, it had a lot of similarities with the NIH campus main entrance. I needed to talk to a security guard so I put my bag down. After he asked me what I did- that is, what I studied, I was surprised to find that he was a scientist too. We had an interesting conversation about science. Then I turned around to get my bag (presumably to enter the building). However, I found that someone had completely taken apart my 35 mm camera while my back was turned- it was entirely in pieces, even the lens was just a pile of glass and black metal and plastic parts. I was shocked, angry, and despondent all at the same time.

I’ve been thinking about this dream all day and it seems to sum up my career stage, my concerns about making it to the next step and succeeding in science, and my concern over the state of science in the US currently- especially during the shutdown. Imagine that the camera represents my vision of science and security represents the grant/career process, especially with an emphasis on funding organizations. Also the security guard? An alternate ending to the career story. The mind is a wonderful and terrible place when it’s worried about something.

What if I were my own post-doc mentor?

Recently I’ve  had time, and reason, to reflect upon what was expected of me during the early portion of my post-doc and what I was able to deliver. It started me thinking: how would I judge myself as a post-doc if I (the me right now) were my own mentor?

My post-doc started 12 years ago and completed when I landed my current job, 7 years ago. I’ve given a short introduction that includes some context; where I was coming from and what I settled on for my post-doc project.

Background: I did my PhD in a structural virology lab in a microbiology and immunology department. I started out solidly on the bench science side then worked my way slowly into image analysis and some coding as we developed methods for analysis of electron microscopy images to get structural information.

May 2001: Interviewed for a post-doc position with Dr. Ram Samudrala in the Department of Microbiology at UW. Offered a position and accepted soon after. My second day on the job, sitting in an office with a wonderful panoramic view of downtown Seattle from tall tower to tall tower, was September 11th 2001.

First idea on the job: Was to develop a one-dimensional cellular automaton to predict protein structure. It didn’t work, but I learned a lot of coding. I’m planning on writing a post about that and will link to it here (in the near future).

Starting project: My starting project that I finally settled on was to predict structures for all the tractable proteins in the rice, Oryza sativa, proteome, a task that I’m pretty sure has never been completed by anyone. The idea here is that there are three classes of protein sequence: those which have structures that have been solved for that specific protein, those that have significant sequence similarity to proteins with solved structures, and those that are not similar to sequences with known structures. Also, there’s a problem with large proteins that have many domains. These need to be broken up into their domains (structurally and functionally distinct regions of the protein) before they can be predicted. So I started organizing and analyzing sequences in the rice proteome. This quickly took on a life of it’s own and became my post-doc project. I did still work some with structure but focused more on how to represent data, access it, and use it from multiple levels to make predictions that were not obvious from any of the individual data sources. This is a area that I continue to work in in my current position. What came out of it was The Bioverse, a repository for genomic and proteomic data, and a way to represent that data in a way that was accessible to anyone with interest. The first version was coded all by me from the ground up in a colossal, and sometimes misguided, monolithic process that included a workflow pipeline, a webserver, a network viewer, and a database, of sorts. It makes me tired just thinking of it. Ultimately the Bioverse was an idea that didn’t have longevity for a number of different reasons- maybe I’ll write a post about that in the future.

Publishing my first paper as a post-doc: My first paper was a short note for the Nucleic Acids Research special issue on databases on the Bioverse that I’d developed. I submitted it one and a half years after starting my post-doc.

Now the hard part, what if I were my own mentor: How would mentor me view post-doc me?

How would I evaluate myself if I were my own mentor? Hard to say, but I’m pretty sure mentor me would be frustrated at post-doc me’s lack of progress publishing papers. However, I think mentor me would also see the value in the amount and quality of the technical work post-doc me had done, though I’m not sure mentor me would give post-doc me the kind of latitude I’d need to get to that point. Mentor me would think that post-doc me needed mentoring. You know- mentor me needs to DO something, right? And I’m not sure how post-doc me would react to that. Probably it would be fine, but I’m not sure it’d be helpful. Mentor me would push for greater productivity, and post-doc me would chafe under the stress. We might very well have a blow up over that.

Mentor me would be frustrated that post-doc me was continually reinventing the wheel in terms of code. Mentor me would push post-doc me to learn more about what was already being done in the field and what resources existed that had similarities with what post-doc me was doing. Mentor me would be frustrated with post-doc me’s lack of vision for the future: did post-doc me consider writing a grant? How long did post-doc me want to remain a post-doc? How did post-doc me think they’d be able to land a job with minimal publications?

Advice that mentor me would give post-doc me? Probably to focus more on getting science done and publishing some of it than futzing around with (sometimes unnecessary) code. I might very well be wrong about that too. The path that I took through my post-doc and to my current independent scientist position might very well be the optimal path for what I do now.

I (mentor me) filled out an evaluation form that is similar to the one I have to do for my current post-docs (see below). Remember, this was 12 years ago- so it’s a bit fuzzy. I (post-doc me) comes out OK- but having a number of places for improvement.

This evaluation makes me realize how ideas and evaluations of “success”, “progress”, and even “potential as an independent scientist” can be very complicated and can evolve rapidly over time for the same person. As a mentor there is not a single clear path to promote these qualities in your mentees. In fact, mentorship is hard. Too much mentorship and you could stifle good qualities. Too little and you could let those qualities die. And here’s the kicker: or not. What you do as a mentor might not have as much to do with eventual outcomes of success as you’d like to think.

SelfEvaluation

How would mentor me rate post-doc me if I had to evaluate using the same criteria that I now use for my own post-docs?

How would mentor me rate post-doc me if I had to evaluate using the same criteria that I now use for my own post-docs?

The 5 stages of reading reviews

In many parts of our lives we have to receive criticism. Sometimes directly, from someone like a boss telling us we screwed up, and sometimes indirectly, in the form of written reviews from anonymous reviewers. In science, reception of criticism, ingestion, and self-improvement as a result are a part of the gig. A BIG part of the gig. We submit papers that get reviewed (that is, criticized) by at least two peer reviewers. We submit grant proposals that get shot down. We present ideas that rub somebody the wrong way- so they tell us in public ways. I’ve had a lot of experience at this. A lot.

Today I found that the renewal of a collaborator’s 30 year old NIH R01 (it’s been renewed 6 times before) that I wrangled myself a co-PI spot on was not discussed in study section. This happens when it gets scored poorly by reviewers and so doesn’t move to the stage of open discussion when the group of reviewers meets. It means that the grant will not be funded and generally that it didn’t make the top 50% of proposals for that round. It stinks.

Here’s how I often react (riffing off of the 5 stages of grief):

  1. Denial. When I first get a poor grant review I often think, “hmmm… that’s weird, there must have been some kind of mistake. I’ll talk to the program officer and get this all cleared up right away”. Loosely translated this means, “my proposal is so good there’s no possible way it could have been not discussed in study section so the only reasonable explanation is that there was a terrible, and highly unlikely, clerical error.” Yeah. Right.
  2. Anger. “Those STUPID nitwits! How could they be sooooo stupid as to not see the brilliance of my obviously brilliant study? What total imbeciles. It’s a good thing that it’s all their fault.”
  3. Bargaining. “OK. You know what, I’ll do better. I’ll do better and write better and experiment better and this will all go away. It has to, right?”
  4. Depression. “I’m a failure and nobody likes me. Also, I can’t do science and I’m an imposter. Everyone else is way smarter than I am. Holy crap what am I going to do with myself now?”
  5. Acceptance. “Right. So I see the points that I need to fix. And I recognize the points that the reviewers just didn’t get. Since they didn’t get them it means that I didn’t communicate them well enough. I can fix this.”

Of course, getting through these to stage 5 is the goal. That’s where the rubber meets the road. How do I take what someone else has criticized me on, strip away the emotional attachment (they’re no attacking me), triage the good from the bad (face it, sometimes reviewers are not paying attention), and apply what you’ve learned to improve what you’ve produced. This process, and the uncomfortable stages that accompany it, has led me to write papers, improved the papers I’ve already written, spawned new ideas, and promoted self realization and betterment. Learning from how others see you is a critical and under-appreciated skill.

 

Time to review for scientific publications revisited

Anna Sharman wrote a couple of excellent posts about time to first response for journals and time to publication after acceptance for journals. Following up my previous post on time spent at the journal (from submission to acceptance) she wrote:

So I went back through my email archive and reconstructed the process for all the papers I previously listed, plus a couple. And I corrected a few inaccuracies in my previous report (I was lumping the two rejections that I then resubmitted anew with their eventually accepted versions- not really fair for the journals). Here are the results which show that the time from submission to first response (in all cases listed except the one with the asterisk this is when I received the first reviews), overall time to acceptance, and finally time to publication after acceptance. The publication time is the first time the article appears on a website since most of these journals have epub before ‘print’ policies. PLoS journals don’t have a physical volume (there is not a physical paper-and-glue PLoS journal) but they do release volumes, collections of articles, at set times.

Overall my reanalysis decreased the mean total time at the journal (from 5.7 months to 4.9 months) and showed that the actual time spent under review (as opposed to time when I was revising the paper according to reviewers’ comments) was about half that- about 2.6 months. I would be interested in seeing if this is typical or not since this is one variable that could be very specific to the way that I work.

The outlier in the analysis is my PLoS One publication that was first considered at PLoS Computational Biology. This appears to have a very short turnaround time, but this is only because the editor at PLoS One evaluated my responses to the reviews received from PLoS Computational Biology and made a decision on that basis.

Finally, this analysis does not take into account several places where the effective time to acceptance was much longer. The aforementioned PLoS One publication was actually submitted to the RECOMB Systems Biology conference where it reviewed well (about 2.5 months) then recommended for consideration at PLoS Computational Biology, where it was reviewed not as well. From start to finish this was close to a year before it was actually published. Likewise the BMC Systems Biology publication that was rejected then resubmitted went through a long process of editorial consideration at the end that extended the time we had it (i.e. shortened the time in review by my calculations) by a lot since we had challenged inappropriate reviews at the editorial level.

The original impetus for the post was

And the current analysis revises my initial assessment since what Nick was really asking about was the turnaround time- that is, the time from submission to receipt of the first reviews for the paper. In this case 100 days is quite a bit longer than normal (as judged by my limited analysis here) since the mean turnaround times I get are about 2 months.

Table. (revised) Survey of time in review for a number of my own papers.

PMID Journal Time to first review Months until acceptance Months spent in review Acceptance to publication
23335946 Expert opinions 1.9 4.7 1.9 0.9
22546282 BMC Sys Bio (rejected) 2.0   2.0  
22546282 BMC Sys Bio (new submission) 1.9 11.2 3.0 1.3
23071432 PLoS CB 3.1 12.2 7.6 1.8
22745654 PLoS One 2.7 6.6 4.2 2.7
22074594 BMC Sys Bio 3.2 3.4 3.3 1.0
21698331 Mol BioSystems 1.8 4.6 3.0 1.0
21339814 PLoS CB (rejected) 1.0   1.0
21339814 PLoS One (new submission *) 0.4 0.4 0.4 0.9
20974833 Infection and Immunity 1.8 1.8 2.2 0.7
20877914 Mol BioSystems 0.8 1.2 0.9 1.4
19390620 PLoS Pathogens 1.8 4.4 2.4 4.6
20974834 Infection and Immunity 2.0 2.9 2.0 0.4
Mean (months) 1.9 4.9 2.6 1.5
  Std dev (days) 24 119 57 35

 

 

 

Twenty years of scientific publication

I just realized that this year marks the 20 year anniversary of my first scientific publication. Wow! That makes me about <mumble mumble> years old. But it’s a good point to reflect on my scientific genesis. The paper (I was middle author) was on biochemical characterization of HIV-1 matrix (MA) protein function, and I contributed by doing some of the work MatrixPaperFig1including showing that MA could be deleted and HIV would still be infectious. We also showed that the deletion-MA HIV packaged the envelope protein differently than the wild-type (naturally occurring) virus. According to Google Scholar the paper has been cited 89 times (here’s a post on how my other papers have fared) and to commemorate its 20th anniversary Journal of Virology has made it available for free on their website. (OK- maybe they just make things available for free after they get really old)

Here’s a timeline for my scientific genesis (the organized, formal part of it- I’ve always been a scientist at heart):

  1. 1989: Took a job for extra money in the Reed biology building supply room. Good move- knowledge and connections.
  2. 1990: Took an upper level bio course at Reed as a sophomore that was over my head (Animal Physiology with Dr. Steve Arch). Way over my head.
  3. 1991: Initiated an independent research project in Animal Physiology to examine the effects of cAMP treatment of Xenopus (frog) embryo neurite outgrowth with microscopy and horse radish peroxidase staining. More successful than it should have been.
  4. 1991: Applied for a summer research internship at Oregon Health Sciences University with Dr. Eric Barklis (see below). Got it somehow.
  5. 1991-1992 (summers): Learned molecular biology techniques, virology, and dipped my toes in how to actually do science.
  6. 1993: Did my Reed senior thesis project (on function of matrix protein) with Dr. Barklis through the year. Graduated from Reed.
  7. 1994-1995: Took a job as a research technician with same Dr. Barklis. Did work, published more papers.
  8. 1995: Accepted to OHSU Department of Microbiology and Immunology for graduate school. Got a pay raise (really) with my grad school stipend. Awesome!
  9. 1996: Chose, surprise, Dr. Barklis as my graduate mentor. Best decision ever.
  10. 2000: Finished grad school, defended my thesis, got my PhD! (that was waaaaaay easier to write than it was to do).
  11. 2000-2001: Did a short post-doc with Dr. Barklis while waiting for my future wife to finish school.
  12. 2001: Accepted a post-doc position with Dr. Ram Samudrala at University of Washington Department of Microbiology doing all computational work. Excellent decision.
  13. 2006: Accepted position at Pacific Northwest National Laboratory in Computational Biology and Bioninformatics. Awesome decision.