Rational Exuberance

Great tips found here:
http://dotnot.org/blog/archives/2005/01/13/move-a-subversion-repository-from-one-machine-to-another/

Quote:

Even if the machines are on different operating systems, this is dead easy. On the source machine, simply ‘dump’ the repository:
svnadmin dump /path/to/repo > reponame.dump
tar zcf reponame.tgz reponame.dump
scp reponame.tgz hostname:/path/to/new/repo

Then login to the new machine, and set up the new repo:
cd /path/to/new
svnadmin create reponame
tar zxf reponame.tgz
svnadmin load reponame < reponame.dump

That’s all there is to it. Then you can of course delete the dump files, the .tgz files, and even the source repo if you are brave.

If you are like me, you have a few sites that are up and running with version 0.6 and they seem to be happy in their stable v0.6 state. You used the PEAR install because really you just wanted an easy way to get started. You built a few sites with it, and now you’d like to jump into the latest version and see what it has to offer.

How do I keep my existing symfony sites happy, but still install the latest SVN version to use with new projects?

First, let’s decide on this: We are going to keep the old PEAR installation just the way it is. The goal is to not have to go back into any existing sites and allow them to keep on working just like they always have been. The problem for me is that I already was playing around with the latest beta by way of “pear upgrade symfony/symfony-beta”. I’ll need to undo this and get my old version back:

#sudo pear upgrade -f symfony/symfony-0.6.3

Notcie the “-f” flag forces the installation of the older version.

Know, Understand, and Love the old PEAR installation.

Just so that I don’t go into this completely blind, I’m going to take a minute and try to understand where the symfony code actually goes when I do a PEAR install. If you run the command “pear config-show”, you should see a list of settings for your PEAR installation. For me, my pear directory is “/usr/local/php5/lib/php”. The old symfony files are all in there. After a bit of investigation I start to recognize the directory structure from the symfony trunk:

/usr/local/php5/lib/php/symfony (”/lib” folder in symfony trunk)
/usr/local/php5/lib/php/data/symfony (”/data” folder in trunk)
/usr/local/php5/bin/symfony (”/bin/symfony” shell script in trunk)

Phew, at least that mystery is solved. Now that I have undone the previous upgrade and have my old version of symfony running the way it was, I need to get my hands on the latest and greatest version from SVN. I don’t want to use the sandbox download because I can’t upgrade it with latest revisions in the repository. So, instead I’ll download a version of symfony from the SVN repository. While doing this I’ll have to be careful to leave my PEAR installation alone.

Download latest version of Symfony from SVN

1. Create a location to download the latest code from the svn repository.
I’m just going to put them in “/usr/local/php5/lib/php/mysymfony”, although you could put them anywhere.

#sudo mkdir /usr/local/php5/lib/php/mysymfony
#cd /usr/local/php5/lib/php/mysymfony
#svn co http://svn.symfony-project.com/trunk/ .

It might take a few minutes to download all of the code into your new folder. Remember, because this is under version control, anytime you want to grab the latest changes to the trunk you can run:

#svn update

from within the “mysymfony” directory and it will update automatically.

2. Create the appropriate link to the new “symfony” shell file so you can run it from the command line.
There are a few ways you can do this, but I’m going to create a symlink to the new symfony file and place it right next to the old symfony shell script:

#sudo ln -s /usr/local/php5/lib/php/mysymfony/data/bin/symfony /usr/local/php5/bin/symfony1.0

What this is basically saying is: “Create a link -symbolically to […]data/bin/symfony and put it in […]/php5/bin/symfony1.0

Now I should be able to call symfony1.0 from the command line and be using the new version.

# symfony1.0 -V
> symfony version 1.0.0-beta2

Let’s try to make a new project using our latest version of symfony:

#mkdir ~/Sites/sf_beta_test
#cd ~/Sites/sf_beta_test
#symfony1.0 init-project sf_beta_test

Success!! I can check and see if it’s going to be using the new symfony libraries by checking in /sf_beta_test/config/config.php. I see it is using the latest lib and data directories:

// symfony directories
$sf_symfony_lib_dir = ‘/usr/local/php5/lib/php/mysymfony/lib’;
$sf_symfony_data_dir = ‘/usr/local/php5/lib/php/mysymfony/data’;

To get my “/sf” directory working in my new project, I have to make sure to set up my host so that knows where to find them. I do this within my httpd.conf file in my Apache/conf directory. (See http://www.symfony-project.com/askeet/1)

<VirtualHost 127.0.0.1>
DocumentRoot [path-to-symfony-project]/web
ServerName sf_beta_test
Alias /sf /usr/local/php5/lib/php/mysymfony/data/web/sf

<Directory [path-to-symfony-project]/web>
Options All
AllowOverride All
</Directory>
</VirtualHost>

Alternatively, I could create a symlink within my app’s web directory called “sf” and point it to the right directory:
#cd ~/Sites/sf_beta_test/web
#ln -s /usr/local/php5/lib/php/data/symfony/web/sf sf

I will probably have to do that once I deploy my project to my shared host on the internet, because I won’t have access to the httpd.conf there! (Unless of course I use the “freeze” command!! More on that elsewhere.)

In summary, I now have two version of Symfony installed on my development machine.

1. The PEAR installation is of the latest stable version, symfony-0.6.3. These files are all in:
“[PEAR directory]/symfony” and “[PEAR directory]/data/symfony”. The binary is in “/usr/local/php5/bin/symfony”

2. The SVN installation is the latest version of the code. It is checked out into “/usr/local/php5/lib/php/mysymfony”.
I have a symlink in /usr/local/php5/bin/ named symfony1.0 that points to the new symfony shell script, originally found in “/usr/local/php5/lib/php/mysymfony/data/bin/symfony”.

Francisco and I were discussing whether or not search-engines had replaced knowledgeable people for certain kinds of questions. Specifically we were interested in the kinds of questions that are more easily answered by people than by google, and what that reflected about the question, the process of answering it, and the person asking the question.

Frisco argued (in my words) that a person’s inability to use a sufficiently capable search engine to answer a question reflects their ignorance of that tool’s featureset, i.e. “Google can answer anything if you know how to ask the question.”

I said that I frequently find that a tight, focused conversation with a trusted human source lasting perhaps only 30 seconds can be more valuable than an hours-long answer-and-question hunting session with google and dizzying thread-tangles of forum-noise.

Following is a dizzying thread-tangle of email about this subject.

Mark Meves Fri, Mar 3, 2006 at 1:42 PM
To: francisco
I might be convinced to share your perpective on this issue, but nonetheless I came accross the conversation with Scott (my brother) in my chat logs and I had a laugh.

Session Start (manikmarkus:scotathon): Tue Feb 14 14:44:19 2006
[14:44] scotathon: mark, what is the function that will take a string and execute it as php, in php?
[14:44] manikmarkus: eval()
[14:44] manikmarkus: right?
[14:44] scotathon: yes
[14:44] scotathon: something like that it impossible to google
[14:44] manikmarkus: careful it’s like the most insecure thing in php.
[14:45] manikmarkus: avoid it when possible. what are you tyring to do?
[14:45] manikmarkus: cause you can dynamically generate property names and method names and class names w/o using eval()
[14:45] manikmarkus: $o = new $class_name();
[14:45] manikmarkus: $o->$some_meth(’blah’)
[14:46] manikmarkus: $age = $o->$property_name
[14:46] scotathon: yeah…
[14:46] scotathon: i’m not sure i want to use eval
[14:46] manikmarkus: $age = $o[$property_name] //same i think
[14:46] scotathon: basically i am loading in an .html file into a string
[14:46] scotathon: $html = file_get_contents(’myhtmlfile.htm’);
[14:47] manikmarkus: right
[14:47] scotathon: but now i find myself needing to put a bit of php in that .htm file
[14:47] scotathon: basically i am emulating smarty
[14:47] manikmarkus: woah. i’ll let you have fun w/ that. be careful
[14:47] scotathon: but i don’t need all that hooha
[14:47] scotathon: i just want to do this:
[14:48] scotathon: $html = file_get_contents(’mytemplate.htm’);
[14:48] scotathon: mytemplate.htm contains:
[Ed: html character entities removed because wordpress broke on them — they got converted to the referred to entity. doh!]

[14:48] manikmarkus: if the mytemplate.htm has php in it, could you require_once() it?
[14:49] manikmarkus: ob_start()
require_once(’file.php’)
$s = ob_get_contents();
ob_end_clean()
[14:49] manikmarkus: perhaps?
[14:50] scotathon: woah, never used ob before
[14:50] manikmarkus: yeah it’s ugly but nec. sometimes
[14:55] scotathon: you rock
[14:56] manikmarkus: sweet glad i could help
[14:56] scotathon: i have no idea how that works, by the way
[14:56] scotathon: require_once doesn’t execute the code?
[14:56] scotathon: or… i guess it does
[14:56] scotathon: but the file can be almost all html
[14:56] manikmarkus: ob_start() says “anything output to the screen, capture it, don’t put it to the screen but start saving it in a buffer
[14:57] manikmarkus: ob_get_contents() says “ok whatever you saved,
give me a copy
[14:57] scotathon: and require_once… will actually “echo” the file contents?
[14:57] scotathon: if it’s html, i mean
[14:57] manikmarkus: ob_end_clean() means “ok stop buffering
(capturing outputted content).
[14:57] manikmarkus: ob_end_flush() means “stop capturing, and output
it to screen
[14:58] manikmarkus: i don’t have much experience require_once()ing non-php files, but php is a language embedded inside other documents (html) so i guess it should work
[14:59] manikmarkus: in the old days, people used to incude() and require() static content all the time: SSI’s
[14:59] scotathon: crazy
[14:59] scotathon: i’ve never done that before
[14:59] scotathon: good times
[14:59] manikmarkus: rite rite. hope this helps . good luck and don’[t refrain from asking me more questions
[14:59] scotathon: thanks
[14:59] scotathon: i appreciate it, you were way better than google
[14:59] scotathon: i’d still be looking
[15:00] manikmarkus: heh
Session Close (scotathon): Tue Feb 14 15:01:21 2006The first person to make a search interface work like that will be a rich man or woman.

I sent this to frisco, and told him this would be a good conversation for my blog. Frisco responds:

francisco Fri, Mar 3, 2006 at 2:00 PM
To: Mark Meves

I see your point. It’s too similar to me asking google how to fix my car’s engine noise. My point is that when i query google for “car engine noise fix” i will get a bunch of pages about how to fix various engine noises. If i do not know much about cars, this will overwhelm me and i will have to read each one carefully and thoroughly, at the end of which i will have a lot more knowledge about cars than when i started, perhaps even enough to actuallly be able to do some good on my car. If (still with limited knowledge) i ask an expert, and the expert says “that noise is due to the 3rd piston needing cleaning” (or something that actually needs fixed on cars, i don’t know enough about them) and i proceed to unscrew everything to get at the pistons, i will end up with a very very broken car.

In your example, do you really feel confident that your brother will write code that works well? [Ed: yes] I feel he would be better off reading through (or at least skimming) the entirety of www.php.net/docs.php and that you just succeeded in giving him the rope to hang himself with.

-f
http://www.blackant.net/

p.s. - i don’t know where your blog is, and can’t think how to query google to find it.

I read the excellent thedailywtf.com for Alex Papadadimoulis’s supreme combination of irony and wisdom when it comes to scrutinizing real-life examples of questionable code — usually code that takes some-or-another methedology or technology to an absurd extreme. (Great examples of XML-overkill, Enterprise-overkill, OOP-overkill, etc.)

While sometimes the site can serve as a tremendous ego-boost to the self-doubting programmer, more often than not I use thedailywtf.com to scrutinize myself — I was first referred to when I was trying to justify the use of a Factory-factory in PHP. (Although after having read the relevant article I still think I was justified, even more so having found a poster that aggreed with me.) (But of course, the internet is good for finding justification for anything, isn’t it?)
Sometimes — but not always — there is useful insight in the 100-or-so comments that follow a posting. Today I enjoyed reading some anonymous poster’s insights on Stored Procedures. He or she articulated my current perspective better than I could have (something else the internet is good for):

Wow, trying to keep application logic out of a database is the *stupidest* thing you’ve ever heard? If there is one property of “enterprise” applications, it is the fact they are often systems that deal with many data sources and services, often performing logic between the disparate sources. But lets list some reasons to keep the logic out of the database.

1) Stored procedures tie you to a particular vendor
2) Application logic in the database has to duplicate logic in the application
3) Horrible source code control (in every case I’ve witnessed)
4) Lack of OO design, hence the term “stored PROCEDURE”
5) Difficulty migrating to a different database vendor/version
6) Puts load on the database, which doesn’t scale/cluster well since it is the data source of record

Instead, put the logic in your objects, keep the database as minimal as possible, and certainly use a good persistence tool to ensure the updates and queries are correct.

I admit, you can instaed take a very data-centric view of things, and many people successfully work under the idea that the database *IS* the application, and everything else is just a GUI/front end. But this is not very scalable, not Object Oriented at all, and IMHO is only suitable for smaller systems, and certainly not large “enterprise” ;) systems. All these points can, of course, be debated…

But wow, this is the *stupidest* thing you’ve ever heard!?! WTF!!!

Thanks Alex, thanks Anonymous.

Given my standard practice of ‘code now, think later,’ (I will call this ‘CNTL’ to rival the other acronyms to describe coding techniques) I tend to write code the quick and dirty way, telling myself that if I need to go back to it later to clean it up I’ll do so when i have the time and it becomes necessary. Of course, I never have the time and rarely is it necessary. I do realize the importance of DRY and code portability, and as my skills as a web developer have improved, I have made the natural progression from learning the programming language to learning programming techniques. I now sit here exploring the web realizing I want to learn about design patterns (especially MVC frameworks, and specifically Symfony and Rails), version control, test-driven development, and IDEs for OS X. Where to begin?
Also, can I get a quick show of hands of how often a developer wants to scrap their whole project and start over? For this very reason I like taking on new projects, because I get a clean slate and get to put into practice all of the new things I learned from my last project.

Lately I find myself asking “What would David Heinemeier Hansson do?” as I code.

I have learned volumes about ruby from reading his source code. Frequently I blindly follow his techniques without necessarily understanding them first — out of character for me. But when it’s DHH, you can be sure that tomorrow you’ll understand it and be glad you did it today.

I almost got up to go for a walk because I’m hitting the 8 hour mark of non-stop coding (no lunch) but then I said out loud “No, David Heinemeier Hansson would stay and finish this.”

I’m making a prototype now for a bracelet inscribed “WWDHHD?” with the intent of getting rich selling it online.

Also: todo: make a video spoofing the OSCON “application in 15 minutes” — but make it like a B2B enterprise system in 20 seconds. And make it all cinematic, with virtual reality gloves and speech to text and lots of shouting and kungfoo poses.

But in all seriousness, thank you DHH. Heros are hard to come by these days. I look forward to the day when I can actually find a shortcoming in your work.

This post is going to serve as a personal reference for when I have to import an existing spreadsheet (in this case it is an excel .xls file) into mosDirectory in Mambo. The listings I have are organized by SIC (standard industry code), and I have a corresponding key that ties SIC codes to category names.

First, we need to look at the table structure for categories. I added the “sic” column so I can keep track of how each category relates to the standardized SIC categories.
The only column that aren’t set to the default values are sic, parent, and name. So, I can take my top level SIC categories and create a query that contains only these fields:

INSERT INTO mos_dir_cat (sic, name, published) VALUES
(1031,’Lead and Zinc Ores’, 1)

Since my list has 3 tiers of categories (we call them Parent Category, Category, and Sub Category), I have to go through a process where I load in the level 1 categories first, load in the level 2 categories (while keeping track of their parent categories in a tmp row), and then load in the level 3 categories while keeping track of their parent categories in a tmp row as well.

1. Load level 1 category names (no sic code)
2. Load level 2 category names, sic code, and parent category names
3. Update level 2 rows, setting the “parent” column with the id of the parent category. To do this we use a SQL to pull the id off of the parent category listed in the “tmp” row:
   

   UPDATE mos_dir_cat as m1, mos_dir_cat as m2
   SET m1.parent = m2.id WHERE m1.tmp = m2.name

4. Repeat the process for level 3, first loading in the names, sic codes, and parent category names (which go into the tmp row), then filtering out the parent category names from the tmp row and inserting the parent id’s into the parent row.

A note on what I mean by “load”: Basically I cut and past the columns I need from the excel document into a text file. It will paste them in with tabs separating the rows. I do a simple search and replace for [\t] (\t stands for a tab) and replace them with [”, “]. Then I search and replace for [\r] (a line break) and replace it with [”),\r(”]. This will turn this:

Agricultural Products - Crops Wheat 0111
Agricultural Products - Crops Corn 0115

into

(”Agricultural Products - Crops”, “Wheat”, “0111″),
(”Agricultural Products - Crops”, “Corn”, “0115″),

Although this involves some manual steps, it is a great way to convert a text and tabs document into simple SQL. I then add my insert statement to the top, and close out the query at the very end of the file by replacing the last comma with a semicolon:

INSERT INTO mos_dir_cat (tmp, name, sic) VALUES
(”Agricultural Products - Crops”, “Wheat”, “0111″),
(”Agricultural Products - Crops”, “Corn”, “0115″),
…
(”National Security”, “International Affairs”, “9721″);

So now that the categories are in (phew!), how am I going to import the 25,000 business into the directory as listings?

1. Organize the columns I need in my excel document.
2. Export as a CSV file.
3. Convert the CSV file to SQL by hand (like I did before when I cut and pasted data from excel into a text file, except instead of converting tabs to [”, “], I just convert the line breaks so that they include the parenthesis at the beginning and end of everyline.
4. Insert these listings into the mos_dir_listings table, leaving the “cid” column blank until the next step.
5. Run and UPDATE query on the mos_dir_listings table, updating cid with the category id using the sic code:
   

   UPDATE mos_dir_listings AS listing, mos_dir_cat AS cat
   SET listing.cid = cat.id
   WHERE listing.sic = cat.sic

6. Man that query took a long time to execute. So long I almost thought it crashed. After making it through all 25,000 rows, we are finished!

We’re done! I now have three tiers of categories and their corresponding sic codes loaded into mos_dir_cat, and 25,000 listings in mos_dir_listings, linked to categories by way of their sic code.

mark,

After out discussion about EZPDO, here is another one that looks more established and is use in the Symfony framework:

http://propel.phpdb.org/trac/

I think I may switch to this because it looks like it’s quite a bit more popular.

Also, I ordered the Ruby book. Doh.

• Creole
• Propel
• Symfony
• Ruby on Rails
• SVN tutorial

[the following is an excerpt from an email i sent:]
Sorry to ramble — I have a question or two at the end.

I checked out your blogs (kiloblog, blogometer) for the first time today.
Enjoyed skimming over them — an activity I rarely engage in.

[note: In the following I ramble on about ideas of what I think blogs “should” do. If there are already blog solutions that implement some of these features, I do not know about them because of my very limited real-world experience with both reading and writing blogs.]

[note: I do not consider any of this my intellectual property. If you like any of these ideas enough to run with them, please do so.]

One of my pie-in-the-sky dreams, as we may have discussed, is to write
blog software that would allow for authentication and then push
relevant entries (or emphasize them) to the reader based on her
profile of interests, or to push entries to different forward-facing
sites. (For example, under this scenario, both kiloblog.com and
blogometer.com would pull their content from the same underlying blog
database.)

Also readers could subscribe using a topic- or content-specific
filter, so that they get notified (RSS’ed ?) only when a post is made
that contains subject-matter or concepts from a set of their
pre-defined interests. (This might be of limited use applied to one
blog author, but applied to a community (or universe) of blogs, I
think this would be a very attractive feature.) I find it very likely
that something like this already exists.

Another cool feature I have daydreamed would be expandible-collapsible
blocks of text that would contain tangential or parenthetical notes.
For example, in a document intended primarily for a non-technical
audience, (like my managers) I would author technical details inside
such a box such that the secondary audience (my coworkers) could read
them, but my managers could easily ignore them.

As an offshoot, imagine writing a status report to a client, but then
also including private details intended only for your a limited
audience (like your development team) all in the same document.

(I am aware that all of these features would require a special
authoring interface that would allow for meta-tagging at such a small
level of granularity. You and I can probably both see the feasibility
of implementing such a thing with javascript.)

I think your “sub-blog” idea (from our conv. early summer ‘05) was
along similar lines. I guess what interests me specifically is the
underlying technical approach to such a scenario.

Riffing off of our conversations also, it would be great to have a
blog entry that is versioned and can revert itself to any arbitrary
version of itself for the reader. (So, for example, you could correct
yourself or build on your own ideas, but still have the earlier
versions of the document available for posterity and reference (which
might come in handy to synch the entry up with a comment that would
otherwise be irrelevant).)

The 37signals product “Basecamp” (free web-based project management)
has a widget they call a “writeboard” that implements such a feature,
but it is not dressed up as a blog.

Despite all this empty brainstorming, to-date i have *no* blog at all
(save for a handful of bad purple-prose entries on myspace) and I am
no closer to implementing the above than I ever have been.

Given that you have far more experience than me, could you please tell
me the blog software you use for your blogs? Do you recommend that
software? We are pretty big into Moveable Type (exclusively, i think)
here at the PR firm. Of course, all of us php developers bemoan its
being written in Perl.

I don’t realistically expect ever to get around to implementing most
of the wacky features I have outlined above. Ironically, I would
rather blog about them and see if someone else ever gets around to it.
;)

–mark meves