It'll be more worthwhile when they start the "baby bullet" service this spring. I wish they'd pick a less misleading name -- the train won't travel any faster, the *only* change is fewer stops. Anyway, with this service you can catch the train in Hillsdale, and the only stops before San Jose are Palo Alto and Mountain View.
The hardest part about Caltrian is getting from the stations to where you really need to go. In my case neither work nor home is convenient, but if this isn't a problem for you then Caltain sounds like a good fit. I've usually used the train on weekends (a service they haven't offered for ~1.5 years), so I don't know what the commute hours are like.