The Peer-Led Team Learning (PLTL) approach has previously been shown to be effective in recruiting and retaining students, particularly under-represented students, in undergraduate introductory CS courses. In PLTL, small groups of students are led by an undergraduate peer and work together to solve problems related to CS. At Columbia University, the Columbia Emerging Scholars Program has used PLTL in an effort to increase enrollment in CS courses beyond the introductory level, and to increase the number of students who select Computer Science as their major, by demonstrating that CS is necessarily a collaborative activity that focuses more on problem solving and algorithmic thinking than on programming. Over the past five semesters, 68 students have completed the program, and preliminary results indicate that this program has had a positive effect on increasing participation in the major. This paper discusses our experiences of building and expanding the Columbia Emerging Scholars program, and addresses such topics as recruiting, training, scheduling, student behavior, and evaluation. We expect that this paper will provide a valuable set of lessons learned to other educators who seek to launch or grow a PLTL program at their institution as well.