The Site Activation Board

In 1965 KSC officials prepared to put it all together at LC-39. After two years of construction, and midway through President Kennedy’s decade of challenge, Kennedy Space Center approached a milestone known in NASA parlance as “site activation.” Two parts of the task were complete: the brick and mortar construction of the facilities, including installation of the utility systems for power, water, heating, and air conditioning; and the electrical and mechanical outfitting such as propellant piping and intercommunications systems. Now came the installation, assembly, and testing of ground support equipment. Earlier chapters have dealt with the first two phases. The third phase in some ways constituted Apollo’s greatest challenge. Hundreds of contractors sent nearly 40,000 pieces of ground support equipment to the Cape for installation at LC-39. On Merritt Island, KSC’s Apollo Program Office had to integrate the activities of more than two dozen major contractors. Engineering and administrative interfaces numbered in the thousands. At NASA Headquarters, Gen. Samuel Phillips, a veteran of the Minuteman site activation program, and his boss, George Mueller, doubted that KSC would have LC-39 ready in time for Saturn V.¹

Rocco Petrone took the first step toward site activation in September 1964 by appointing Lt. Col. Donald R. Scheller “Staff Assistant for Activation Planning.” Scheller, a B-17 pilot in World War II, had just completed four years with the Atlas Missile Project Office. An October 1964 memo from William Clearman’s Saturn V Test and Systems Engineering Office listed the responsibilities of Scheller’s new position. He was to analyze:

• Construction schedules of facilities under the cognizance of the Corps of Engineers.
• Delivery schedules of all ground support equipment to be installed on complex 39 regardless of the source of the equipment.
• Tests to be performed on the facilities by contractors prior to release to KSC as well as the tests on utilities, subsystems, and systems to be performed after these facilities are accepted by KSC.
• Tests that are to be performed under the direction of KSC personnel after ground support equipment is installed.²

Drawing upon the support of KSC’s various design and support elements, Scheller was to develop a work schedule for site activation. His plans would become the management tools to accomplish the task efficiently. After KSC began implementing the site activation plans, Scheller would prepare facilities description documents for LC-39.

Although Clearman was projecting no mean task, he underestimated the job. Scheller took several months to review the situation before organizing a Site Activation Board in March 1965. At the board’s first meeting, he outlined his plans to 40 NASA and contractor representatives. The Site Activation Board, under the aegis of the Apollo Program Office, would work at the management level of KSC and the stage contractors; subordinate groups would handle daily site activation problems. The board was not to usurp other organizations’ responsibilities.

Scheller’s subordinates presented a performance evaluation and review technique (PERT) for the LC-39 activation. PERT schedules would provide three levels of control. At the A level, PERT would focus on the major control milestones for the Saturn V program, e.g., the first facility checkout of the Saturn V test vehicle, SA-500-F. B networks would track each major element required to support the key milestones, e.g., firing room 1 for 500-F. Level C networks, providing a further breakdown of B-level networks, would follow the progress of all subsystems within each major facility, e.g., the propellants loading panel in firing room 1. The Schedules Office, supporting the board, would maintain the A and B levels, while NASA line organizations and stage contractors prepared the C networks.³

The PERT networks brought order to LC-39 site activation. PERT defined each task, performer, and deadline in a descending and expanding level of detail. The top or A network also served as the site activation master schedule, establishing major milestones. This master schedule, prepared by Scheller’s office, was divided into segments or “flows.” Flow 1 charted the activation of the minimum facilities and equipment necessary for the checkout and launch of the first Apollo-Saturn V vehicle, AS-501. A preliminary objective was the arrival and erection of the facilities checkout vehicle, SA-500-F, which would be used for testing and validation of launch facilities and operating procedures. Flow 1 listed as minimum facility requirements:

• Mobile launcher 1
• Crawler-transporter 1
• High bay 1 in the assembly building
• Firing room 1 in the launch control center
• Mobile service structure
• Launch pad A
• Propellants and high-pressure gas facilities
• Related mechanical equipment, electrical-electronic support equipment, and other ground support equipment.

Flow 2 charted the activation of additional facilities to support AS-502, including launcher 2, high bay 3, firing room 2, and related ground support equipment. Flow 3, originally intended for AS-503, tracked the activation of pad B and related facilities such as the crawlerway. A fourth flow covered the remaining LC-39 facilities.⁴

B and C networks supported each of the flows. The B networks eventually listed over 7,400 events, e.g., completion dates for equipment installation. These events covered all facilities and some major components within the facilities. The C networks, largely a contractor responsibility, listed 40,000 activities in sequence and set the dates by which one contractor would have to complete a job to make way for the next operation. A numbering system facilitated the transposition of data between C and B levels, matching as many as 15 C-level activities with their B counterparts.

Following the PERT description, the Site Activation Board discussed a second management tool, the equipment records system. NASA was compiling in Huntsville a list of 40,000 pieces of ground support equipment, the data coming from the engineering divisions of the three spaceflight centers. The lists provided: a name and number; an estimated-on-dock date, the expected delivery date at KSC; and a required-on-dock date, when KSC needed the item for installation. The board intended to use the equipment records system as the communications medium between KSC users and equipment suppliers. Representatives of the Facilities Engineering and Construction Division reported on the current status of key construction milestones, including rack and console installation. All agencies were asked to review the construction status in terms of their organizational needs for access, and to report any problems to the Site Activation Board. Scheller requested comments on the board charter and PERT networks within a week and an early submission of level C data.⁵

Under its charter, the board was responsible for ensuring that all facilities and support equipment comprising the Apollo-Saturn V operational launch base were “constructed, outfitted, installed, interconnected, and tested” in preparation for subsequent operations. This included equipment modifications during site activation. KSC division chiefs quickly expressed concern about these broad powers. In his comments Dr. Hans Gruene asked:

“Will decisions of the Board be made at the discretion of the chairman or by some other method?... Can a decision be appealed by the director of an operating unit [such as the Assistant Director for Launch Vehicle Operations] to the Director of Plans, Programs, and Resources [Petrone]?”⁶ Col. Aldo Bagnulo, acting Assistant Director for Engineering and Development, not wanting the board to assume any of his responsibilities for facility development, said: “The recent emphasis on performing work through normal procedures rather than by committee action should be followed.”⁷ Raymond Clark, Support Operations Chief, raised the same sensitive issue: “Additional clarification is needed as to the depth of management control anticipated by the Site Activation Board.”⁸ Despite these objections, Petrone had his way; when the board began operations, it enjoyed a wide-ranging authority.

The preparation of PERT schedules monopolized the board’s attention for several months. The fashioning of the detailed C-level networks proved time-consuming, and when submitted, data from the contractors forced revisions in the B networks. KSC also had trouble bringing the equipment records system under its control. By August 1965, however, both that system and PERT were computerized and operational. Early that month Scheller initiated biweekly board meetings.⁹ In early October the board moved to new quarters in firing room 4 of the launch control center. Even desks and telephones were in short supply, but KSC got on with the installations, and the following month the board had work space, conference areas, and a management information display and analysis room. The display brought home the immensity of the board’s task. Magnetic devices on a 21 × 5-meter metal wall revealed the status of each PERT chart and told the story of site activation. Tiered seats accommodated 90 people with standing room for another 50. Four rear-projection screens, above the metal wall, provided simultaneous or selective viewing of activation data. Apollo officials could view level A networks, milestone event charts, and major problem summaries on two rear-lighted display areas located to the right and left of the four screens. As activation moved into high gear, the display room was used to brief visiting dignitaries on program goals and progress. Level C networks had an area of their own behind the huge display wall. The Site Activation Board laid out the 40,000 events of the C networks on 418 square meters of metal wall space. A Boeing team, responsible for updating the network, worked at nearby desks. Offices and a graphics section occupied the rear section of the firing room.¹⁰

Site Activation Working Groups

During its first months, the Site Activation Board functioned more as a working group than a management team, but in September Scheller activated several subordinate groups. Lt. Col. Richard C. Hall took command of the Site Activation Working Group, set up to resolve technical interface problems and devise methods of accomplishing new requirements within a given facility. At its first meeting in December, Hall introduced a typical problem. The Communications Service Branch had not received telephone requirements for firing room 1 from the operating organizations. Hall asked the group to submit all requirements at least 60 days prior to the “need date” (the date on which an item was required). The following month Hall’s group assumed formal responsibility for daily site activation matters.¹¹

James Fulton, Launch Vehicle Branch chief in Clearman’s office, recruited Donald Simmons to handle LC- 39’s electrical cable problems. Simmons’s experience on Atlas served him well as the first chief of the Cable Working Group. The group’s mission in September 1965 involved preparation of a cable accounting system, the Site Activation Board’s third essential management tool. This tracking system kept tally on more than 60,000 cables including all connectors by part number, the length of cable, cable makeup, procurement action and date, the agency furnishing the cable, the need date as assessed from the PERT schedules, “from and to” locations, and the installation contractor. Communication and instrumentation from the launch control center to pad B alone required nearly 160 kilometers of cable. KSC let a $2 million contract for the job in October 1965; the work included the installation of 142 kilometers of coaxial, video, telephone, and instrumentation cables plus terminal equipment. The group managed network configuration through computer printouts and network diagrams, with a General Electric team in Huntsville preparing the cable interconnect drawings.¹²

A contractor and two working groups played important logistical roles in site activation. The Boeing Company, primary integration contractor on the Minuteman program, gathered, processed, and reported data for the Site Activation Board. While much of Boeing’s effort involved the PERT schedules, its management systems staff at the Cape effected major improvements in the equipment records system. During the fall of 1965, few engineers relied on the system. When someone needed information about ground support equipment, he normally went to the designer. In early December the equipment records system lacked nearly 33% of its essential data; 79% of the support equipment did not correlate with a PERT activity. Boeing initiated a four-month search and classification program that reduced the respective figures to 5% and 7% and made the equipment records system an effective tool.¹³

The Equipment Tracking Group benefited from the resulting improvements. This group resolved differences between the estimated-on-dock and required-on-dock dates and tracked all items until installation and final testing. The group reflected Colonel Scheller’s belief in management by exception, concentrating on items that failed to meet schedule dates or arrived in the wrong configuration. When this happened, team members scrambled to devise acceptable “work-around” measures. ¹⁴

Interface Control Documentation

Interface control documentation, an essential activity during site activation, was another responsibility of KSC’s Apollo Program Office. Apollo configuration control dated from February 1963, when the manned spaceflight centers had agreed to consolidate and store interface control documents. During the next several years, Apollo-Saturn subpanels placed hundreds of such documents in a Huntsville repository. Through the interface control documents, Apollo managers made sure that thousands of items, built in many different places, would fit and work together. The documents provided design requirements and criteria for hardware and software interfaces, describing the parameters and constraints under which the interfacing items functioned. The information in the documents varied and might include physical and functional design details and operational and procedural requirements. Where an interface involved two NASA centers, a level A document applied - for example, the interface between a command module (Houston responsibility) and the mobile service structure (KSC responsibility). Level B documents pertained to intra-center interfaces such as the S-IVB-Instrument Unit interface covered by Marshall’s Saturn ICD, 13M06307 (October 1965). When changes affected performance, cost, or schedule accomplishment, the centers prepared interface revision notes.¹⁵

Although the Panel Review Board (established in August 1963) gave NASA Headquarters limited control over configuration decisions, General Phillips provided the centers with detailed directions in his May 1964 Apollo Configuration Management Manual. The manual, patterned after Air Force procedures, included a requirement for Configuration Control Boards at each center. KSC had difficulty fitting Phillips’s management scheme onto a program already under way. In September 1965 however, Petrone announced plans to implement it. Maj. Andrew Reis’s Configuration Management Office would “interpret the requirements of [the manual] and define the degree of flexibility necessary to integrate KSC operations consistent with the requirements of Configuration Management.”¹⁶ Petrone’s directive also established a series of Configuration Control Boards, or change boards as they were usually called. Edward Mathews chaired the Saturn IB board; William Clearman, the Saturn V board; and Hugh McCoy, the spacecraft board.¹⁷

Apollo-Saturn subpanels continued to prepare interface control documents and notes. When inter-center panel representatives reached technical agreement on an interface requirement, the proposal would go to an appropriate change board. The board would circulate a “request for impact” through KSC to ensure that the proposed document had no adverse impact on any center function. Other details solicited by the change board included the cost of modifications and the “need dates” of operations and maintenance groups. The Configuration Management Office served as a secretariat for the change boards. When a proposal proved acceptable, the board would notify the other centers to implement the document.¹⁸

Since unapproved interface control documents left open the possibility of an unsatisfactory interface, program offices made strenuous efforts to coordinate their work. Nevertheless a backlog of “open” documents had developed by 1968 that gave NASA officials much concern. A Boeing investigation in May 1968 found two weaknesses in KSC’s program: the documents contained extraneous material that made inter-center coordination difficult, and the complicated processing wasted time. KSC’s program office overhauled its procedures during the next six months and closed out all control documents before the first manned launch of an Apollo-Saturn in October 1968.¹⁹

500-F-A Dress Rehearsal

The Site Activation Board focused its attention in the fall of 1965 on the 500-F test - a dress rehearsal for the new Saturn V rocket and launch complex 39. Plans for the 500-F test vehicle dated back to early 1962 when LOD engineers were still studying the use of barges to move the giant Saturn V to its launch pad. As the facilities checkout vehicle, 500-F would test the mating of the stages in the assembly building, the fit of the service platforms, the launcher-transporter operation, the propellant loading system, and the test connections to the mobile launcher and support equipment. Each dummy stage would duplicate the flight configuration, ordnance, and umbilical connections of its live counterpart. Although inert, the retrograde rockets, ullage rockets, and shaped charges would have the dimensions of the live ordnance. This allowed the launch team to practice ordnance installation. Facility checkout would culminate with a “wet test” to verify the storage and transfer of the propellants. The wet test would involve hundreds of components: pneumatic valves, liquid sensors, time delay relays, pressure switches, circuit breakers, pumps, motors, fans, vaporizers, vents, and the burn pond. The launch team scheduled the delivery of the 500-F stages at the Cape nine months before the first Saturn V flight. The Office of Manned Space Flight translated this into a tentative July 1965 test date.²⁰

This was not to be. When George Mueller revised the Apollo schedule in November 1963, erection of the SA 500-F stages on the mobile launcher slipped back to 1 February 1966. Marshall would deliver an S-IVB- F stage (used in the Saturn IB checkout of LC-34) in May 1965, and the S-IC-F and S-II-F stages in January 1966. General Phillips announced the Apollo launch schedule in February 1965, as follows:

Although planners were dubious about meeting the AS-501 launch date, two more Saturn V launches were scheduled for 1967. From the start of the 500-F test, KSC would have nearly a year to prepare for the first Saturn V launch.²¹

There was little margin for error. In December 1964, Dr. Arthur Rudolph, Saturn V Program Manager in Huntsville, asked KSC to agree to a delay in delivery dates for the 500-F and AS-501 launch vehicles. After reviewing the schedules for equipment installation and checkout, the 500-F test, and AS-501, Petrone replied that there was no room for delay. KSC had already eliminated the detailed receiving inspection for the 500-F and 501 vehicles. Although Marshall’s contract with Douglas omitted the digital data acquisition system test for 500-F propellant loading, KSC would not waive this check. The schedule did include several weeks of learning time, primarily in crawler operations with a space vehicle aboard the mobile launcher. Petrone, however, considered the 500-F schedule “optimistic since it does not allow time for resolution of major difficulties which may occur.”²²

The Crawler-Transporters Begin to Crawl

Events were to reveal a little slack in the LC-39 activation schedule, just enough to recover from a near disaster. The crawler was the prima donna of the Site Activation Board drama of 1965. This gargantuan tractor, designed to carry the 36-story Apollo-Saturn V space vehicle from the vehicle assembly building to the launch pad, caught the public eye; no other facility, excepting the assembly building, got like publicity. Perhaps on account of the public interest, the crawler engendered a series of labor and political disputes, as well as mechanical problems, that nearly disrupted the site activation schedule.

The Marion Power Shovel Company built the two crawlers in Ohio and then took them apart for shipment to the Cape. Under its contract, Marion intended to reassemble the crawlers on Merritt Island with an Ohio work crew, members of the AFL-CIO United Steelworkers. The Brevard (County) Florida Building and Construction Trades Council, citing the Davis-Bacon Act, insisted that on-site construction fell under its jurisdiction. The local unions won a Department of Labor decision in August 1964, but agreed to a compromise that let the Marion crew remain on the job. Although the labor dispute simmered throughout the winter, W. J. Usery and the Missile Site Labor Commission managed to avert a major shutdown. On the basis of the labor difficulties, Marion won a delay in the crawler testing date from November 1964 to late January 1965.²³

The crawler moved under its own power for the first time on 23 January. NASA officials observed that “the initial crawler-transporter was not in a state of complete assembly ready for joint testing” and forwarded a list of deficiencies to Marion.²⁴ Additional runs in April tested the propulsion and steering systems. On the 28th Gunther Lehman of Marion drove the crawler about 900 meters at a speed of 1.1 kilometers per hour; this was a “press day” ride with Debus, Petrone, and other KSC and Marion Power executives aboard. The hydraulic jacking and leveling system was ready for testing on 22 June when the crawler picked up its first load, a mobile launcher. Although the test was labeled a success, the launch team noted high hydraulic pressures when the crawler trucks scuffed on the crawlerway during turns. The treads also chewed up large portions of the macadam surface.²⁵

For Want of a Bearing

On 24 July the crawler moved a launch umbilical tower about 1.6 kilometers to test the crawler on two short stretches of road, one surfaced with washed gravel (”Alabama River rock”) and the other with crushed granite. Preliminary data on steering forces, acceleration, vibration, and strain pointed to the gravel as the better surface. While the crawler was making its run, members of the launch team found pieces of bronze and steel on the crawlerway - the significance of which was not immediately recognized. The transporter was left out on the crawlerway over the weekend because of problems with the steering hydraulic system. On the 27th more metal fragments were discovered and a thorough search disclosed pieces of bearing races, rollers, and retainers from the crawler’s traction-support roller assembly. After the transporter was returned to its parking site, a check of the roller assemblies revealed that 14 of the 176 tapered roller bearings were damaged. KSC engineers attributed the failure primarily to thrust loads encountered during steering; the anti-friction support bearings, about the size of a can of orange juice concentrate, were underdesigned for loads exerted during turns. For want of a bearing, the crawler was grounded indefinitely. And for want of a crawler the site activation schedule and the entire Apollo program would be seriously delayed.²⁶

A reexamination of Marion’s design calculation indicated some other significant facts. The designers had assumed an equal load distribution on all traction support rollers; perfect thrust distribution over the entire bearing, i.e., an axial thrust equivalent to the radial load; and a coefficient of sliding friction of 0.4 (meaning it would take four million pounds of force to move a ten-million-pound object). During the early crawler runs, KSC engineers discovered an unequal load distribution on the traction support rollers. At times as many as four of the eleven rollers on one truck were bearing no load. The thrust, or side load, proved greater than expected. Finally, the crawler tests revealed that the estimated coefficient of sliding friction was far below the actual resistance experienced on the crawlerway. At a crawlerway conference on 27 June 1963, NASA engineers had insisted on a minimum design coefficient of 0.6. In the first runs on the crawlerway’s macadam surface, the coefficient reached nearly 1.0.²⁷

Troubles with the crawler had not been unforeseen. Prior to the roller bearing crisis, M. E. Haworth, Jr., chief of the KSC Procurement Division, upbraided Marion for making difficulties about the tests:

KSC has tolerated innumerable delays in the assembly, tests and checkout operations of CT-1. These delays are to the definite detriment of Apollo facilities readiness and Marion’s position as to the testing operations, will, if carried out, likely cause even further delays which will have a definite and substantial dollar impact on other projects directly and indirectly connected to the crawler transporter concept. The failure of Marion to fulfill its delivery obligations is in itself costing the government substantial sums which were not contemplated.²⁸

On 14 October 1965 Haworth wrote Marion, expressing grave concern over the inactivity at the erection site consequent on a new labor dispute (the unions stayed off the job for nearly six weeks). The roof fell in on both NASA and Marion when the bearing story reached the press and television. Walter Cronkite told his evening newscast audience that the crawler was sitting on wooden blocks under the hot Florida sun, with a top Washington official stating privately that it might never work. The press and Cronkite revived the controversy over the award of the contract to Marion. Politics, they hinted, was involved; and in any case the low-bid procedure might prove penny wise and pound foolish.²⁹

NASA and Marion could answer that the design and construction of a land vehicle expected to carry 8,000 metric tons was without precedent. Its very size, as the Corps of Engineers had pointed out, ruled out preconstruction tests of the coefficient of friction in its moving components. A more pertinent answer was to develop a new bearing, a hydraulically lubricated sleeve bearing made of Bearium B-10. KSC selected the bronze alloy after testing a half-dozen materials at Huntsville. The new design provided separate bearings for axial thrust and radial loads. KSC retained in the design the original supporting shafts that housed the bearings. Although the sleeve bearings would not reduce the amount of friction, they would eliminate the possibility of a sudden, catastrophic failure. Periodic inspection could determine the rate of wear and need for replacement. The disadvantages of the sleeve bearings-lubrication difficulties, the inability to predetermine useful life, and a need for more propulsive power because of increased friction - were acceptable. Fortunately, while the crawler design had underestimated friction, there was a considerable reserve of power. At KSC and Marion, engineers designed a new bearing system. A parallel effort modified the crawler’s steering hydraulic system, almost doubling the operating pressure. At KSC, the burden of the bearing crisis fell principally on Donald Buchanan’s shoulders. In Marion, Ohio, Phillip Koehring directed the redesign.³⁰

Marion reinstalled the support roller shafts in early December. A prototype of the sleeve bearing arrived on the 14th. After cooling it in dry ice and alcohol, the assembly crew placed the bearing in its housing. The fit proved satisfactory, and the remaining bearings were installed by mid-January. On 28 January 1966, the crawler transported a mobile launcher approximately 1.6 kilometers to the assembly building. Bearing measurements indicated an acceptable heat factor. Fortunately, KSC had initiated the crawler contract early enough to allow for both labor disputes and redesign of the bearing.³¹

“Negative Slack” in “Critical paths”

The nerve center for site activation lay in John Potate’s scheduling office. Potate, a young engineer from Georgia Tech, had previously worked on site activation for LC-34 and LC-37 and brought that experience to his new and much bigger assignment. Supported by a Boeing team, Potate put the B PERT network to use. The scheduling office had little control over the A networks; the Apollo Program Offices at NASA Headquarters and KSC set the key milestone dates. The line organization level-C networks provided valuable data, but were too detailed for quick program evaluation. KSC officials, in large part, based their program decisions on the B networks - the level where Potate reconciled C-level capabilities with A-level deadlines. Potate relied heavily on PERT to identify problems. A computer, after processing all available network data, printed out “critical paths,” which traced the controlling chain of events leading to a goal. For each critical path, the computer sheet also indicated the “negative slack.” That curious term indicated how far a facility’s development lagged behind its readiness date; or, put another way, how far behind schedule the Site Activation Board was in meeting a particular goal. The critical paths, consolidated into the PERT analysis reports, became the focal point of board meetings. Potate and the board examined the negative slack in each critical path and searched for ways to eliminate it.³²

Some critical paths showed more than a year’s negative slack when the board started work in August 1965. One involved the mobile service structure, under construction for only six months and nearly a year behind schedule. Since there would be no spacecraft tests during the initial checkout of LC-39, the absence of the service structure would not affect the 500-F schedule. If the service structure continued to lag, however, it would delay the AS-501 mission, the first Saturn V launch. At the board meeting on 5 August, the engineering directorate reported new efforts to speed up development of the service structure. A Marshall representative acknowledged that the electrical support equipment was a week behind schedule. At the moment LC-34 had priority, but the LC-39 electrical equipment dates would somehow be improved. The Corps of Engineers disclosed that the assembly building’s first high bay would not be ready for use by its scheduled date, 1 October, without accelerated funding. Since the October date impinged directly on 500-F, the board agreed to spur construction. In one piece of good news, Bagnulo’s engineering representative announced a “work-around” schedule for the mobile launcher’s swing arms. Hayes International of Birmingham would proceed with the late delivery of the first set of service arms to Huntsville. When checkout there had been completed, the arms would be moved to Florida for 500-F. Where there was insufficient time for testing, Hayes would ship the corresponding arms from the second set to KSC. After the latter had satisfied 500-F needs, KSC would exchange them with Marshall for the first set, which would have been tested by then.³³

Although KSC managed to occupy portions of the first high bay (floors 1-7) on 1 October, the status of other facilities continued unsatisfactory. The construction firm of Morrison-Knudson-Hardeman expected to complete structural steel work on the mobile service structure in late November. This would leave eight months for the installation of ground support equipment, spacecraft piping, and instrumentation and communication cables. At the board meeting on 28 October, John Potate reported 40 weeks of negative slack in the service structure. KSC could eliminate 75% of the lag by performing some necessary modifications during the installation phase. The remainder involved the installation and testing of spacecraft checkout equipment. Potate asked the North American representative to determine what portion of his requirements could be accomplished at the erection site rather than on the pad. Scheller appointed a NASA group to consider ways of shortening the cold flow and hot flow tests. In the former, engineers validated the spacecraft hypergolic systems with nontoxic freon, testing pumps, umbilical lines, and pressure valves; the latter test employed the toxic hypergolics used in flight.³⁴

Huntsville’s electronic support equipment continued to lag behind schedule, and in early November Marshall announced that it would miss the 7 February 1966 deadline for final deliveries by three months. Petrone protested to General Phillips:

This results in a completely erroneous representation of activation constraints and precludes accurate status assessment and realistic planning. In order to use the KSC LC-39 PERT system effectively, and obtain credibility in the eyes of the users, I must reflect true status of MSFC [electronic support equipment] into the networks.³⁵

He asked to revise the PERT charts in line with MSFC’s May date. Although 500-F erection did not require the electrical support equipment, subsequent tests would use it. Petrone sought a new erection date of 15 April 1966.

Phillips’s response set a deadline of 15 April for the delivery of the electrical support equipment and the erection of the test vehicle. On 7 December Petrone explained to Phillips how the new stacking date would affect operations. Of the nine swing arms, only seven would be installed by 15 April. While Marshall would have qualified only three of the seven, KSC would use substitutes from the second set. The launch team would install the service module and command module arms after 500-F erection and before its transfer to pad A. Petrone noted that the mid-April date allowed sufficient time to accomplish modifications to the assembly building and resolve mobile launcher-vehicle assembly building interface problems. The new date also placed some constraints on KSC. Assuming arrival of the AS-501 vehicle stages in early September 1966, the 500-F tests would require six-day, two-shift operations to prevent overlap with launch preparations for the first Saturn V. Even with the accelerated schedule, KSC faced the unpleasant task of installing qualified swing arms concurrently with AS-501 erection. Phillips accepted the mid-April schedule with the understanding that KSC would try to advance the date.³⁶

While Petrone’s office shuffled dates, rumors of another problem disturbed KSC leaders. Workers on pad A believed the foundation was sinking. The charge was serious; excessive settling might damage pipes that serviced the pad from nearby facilities. At three successive meetings Colonel Scheller pressed Steven Harris, the Engineering Division’s site activation chief, for a detailed status report. In mid-November 1965, Colonel Bagnulo responded with reports to Debus and Petrone. There was minor, nonuniform settlement at the pad, but this lay within tolerance. After the Gahagan Dredging Company had removed some 24 meters of surcharge from the pad area in mid-1963, the soil had risen about 10 centimeters. This rebound, which represented the soil’s elastic action, was expected to be the limit of any resettlement after pad construction. Measurements in July 1964, after pouring the 3.4 meters of concrete mat, indicated a maximum settlement of 9 centimeters. Settlement at the sides of the pad varied as much as a half centimeter. As recent measurements by the Corps of Engineers showed little change, the Engineering Division concluded that the launch pad was attaining a stable condition.³⁷

Petrone’s office advanced the activation schedule early in the new year as Marshall accelerated the flow of electrical support equipment to LC-39. At the 6 January meeting of the Site Activation Board, Scheller asked all organizations to consider the feasibility of moving the mobile launcher into high bay 1 on 28 January and beginning 500-F erection on 15 March. The discussion pointed up some confusion on the date for mating the crawler and mobile service structure. While the service structure contract indicated mid-July, the Launch Vehicle Operations Division preferred to delay it until completion of the 500-F tests on 1 September. Scheller indicated that the board would resolve the matter after consulting with Launch Vehicle Operations and Spacecraft Operations.³⁸

The crawler brought mobile launcher 1 into the assembly building on 28 January, meeting the first major milestone of flow 1 (table 6). In the four months since the occupation of high bay 1, the stage contractors had outfitted their respective service platforms and the adjacent rooms.³⁹

The Apollo team had no time to celebrate its accomplishment; PERT analysis showed considerable negative slack toward the next major milestones. Potential delays existed in the delivery and installation of service arms and electronic equipment from Huntsville, the installation and checkout of the operational TV system in the launch control center, and the delivery of spacecraft checkout equipment for the mobile launcher and mobile service structure. At the Site Activation Board meeting on 3 February, Potate noted that PERT dates for erecting the 500-F were three weeks behind the scheduled milestone; the PERT dates for power-on and the pad A wet test lagged eight weeks. The 500-F delays in turn set back the ready dates for the first Saturn V launch. Although North American Aviation expected to be at least 13 weeks late in delivering the first S-II stage, General Phillips’s office continued to list the AS-501 mission in January 1967. If the site activation team did not want the blame for holding up that launch, the PERT dates would have to be improved.⁴⁰

Potate’s review did not include KSC’s newest emergency. When a subcontractor declared bankruptcy in December, American Machine and Foundry Company found itself without cables for its tail service masts. The delivery date for the masts was only four months away and the company still faced difficulties with hood fabrication, tube bending and flaring, and cleaning facilities for the mast lines. Furthermore, contract losses were rumored to exceed $500,000. At a meeting on 1 February 1966, American Machine and Foundry refused to accept new delivery dates since it had outstanding time and cost claims against 24 NASA change orders. KSC responded quickly, removing the cleaning requirement from the contract and dispatching technicians to the York, Pennsylvania, plant. In mid-February Bendix accepted responsibility for completing the cables. A NASA report from the York plant, however, painted a bleak picture. Contract losses on the tail service masts were disheartening. A recent Navy bomb contract offered large profits and the company, understandably, was concentrating on this project. According to the NASA observers, the company was also hiding behind the cable problem “assuming that a subcontractor would require weeks to produce them so they did not proceed with the production, cleaning, and assembly of tubing and components.”⁴¹

KSC’s concern apparently impressed American Machine and Foundry management, because the company voluntarily changed to two 12-hour shifts and a seven-day work week. In return NASA reconsidered American Machine’s claim of additional costs. Despite the settlement, it seemed unlikely that the York plant could deliver the masts in time for 500-F’s power-on date in May. KSC improved the likelihood through several work-around agreements. The center installed certain equipment at Merritt Island rather than at Huntsville, postponed line cleaning until after the power-on exercise on 13 May, and deleted the installation of vehicle electrical cables from the American Machine and Foundry contract. Gruene’s group accomplished the latter task at KSC after the tail service masts had been mated to the mobile launcher. The shortcuts allowed the launch team to install the three masts by mid-April.⁴²

TABLE 6. SLIPPAGES IN LC-39 SITE ACTIVATION,
20 JANUARY 1966

Six service arms for the Saturn arrived by mid-March. In the transfer aisle of the assembly building, Pacific Crane and Rigging crews mounted two hinges to each swing arm. The hinges, 1.2 meters high and 2.1 meters wide, required careful alignment so that the arms would hang and retract properly on the mobile launcher. When this task had been completed, the 250-ton crane lifted each arm into high bay 1 where an eight-man rigging team secured the appendage to the launcher. The operation was expected to take 16 hours, but late deliveries forced the riggers to speed up their work. On 12 March, three days before the first stage was erected, they hung arms 1, 2, and 5. Mounting the sixth arm, 61 meters above the mobile launcher base, was a particularly impressive job, with riggers leaning from the top of the hinge to secure bolts in the tower. After the arms were hung, Pacific Crane ironworkers (craft unionists) routed the umbilical lines from the tower consoles to the swing-arm interface plate. Pipefitters and electricians (industrial unionists), employed by the stage contractors, then took over, routing the umbilical lines along the service arm to the launch vehicle.⁴³

The rigging teams, supervised by KSC’s Richard Hahn, faced another difficult task when the last swing arm for the Saturn arrived from Birmingham on 15 April. Four days later KSC mounted the arm, carefully working it into the narrow space between the tower, the 500-F vehicle, and the two adjacent arms. The late delivery threatened to delay the start of the power-on exercise, a little more than three weeks away. Potate solved the problem by rescheduling Boeing and North American activities in conjunction with the swing arm work of Pacific Crane and Rigging. In a rare spirit of cooperation, industrial union and craft union members labored alongside nonunion workers and civil servants to eliminate two weeks of negative slack. The last swing arm was ready on 8 May.⁴⁴

500-F Up and Out

KSC passed its second major hurdle in March, erecting the 500-F launch vehicle in high bay 1. Crane operators began practice runs in February, using a 9.5-meter spherical water container. Stanley Smith, Bendix senior engineer for the crane and hoist group, simulated the different weights of the Saturn stages by varying the amount of water. On 15 March the 250-ton crane lifted the 500-F first stage from the transfer aisle to a vertical attitude and up 59 meters. After moving the S-IC-F stage through the opening in the bay trusswork, the crane operators lowered it gently to the platform of the mobile launcher. The second stage, S-II-F, followed the same route on the 25th, when it was mated with the first stage. The third stage and instrumentation unit joined the stack before the end of the month. Concurrent exercises out on pad A tested the interfaces between mobile launcher 3 and the pad. The final test pumped 1,135,000 liters of water through the deluge system on the launcher.⁴⁵

The site activation schedule allowed three weeks in April for GETS (Ground Equipment Test Set) tests. The exercises, a long standing Marshall policy, verified Saturn V ground support equipment before its initial hookup with the launch vehicle, in this case the 500-F test vehicle. The checkout also acquainted stage-contractor crews with the equipment. Problems with the Brown Discrete Control System and the Sanders Saturn V Operational Display System threatened to delay the start of the GETS tests on 8 April. The Brown equipment, located in the launch control center, controlled the flow of signals into and out of the RCA 110 computer in the launch control center. The Sanders system in the firing room ran a series of consoles that displayed data from the control center computers for operational use. KSC technicians had the display systems working by 1 April.

The first week of GETS tests featured power and network checks of all electrical support equipment. The following week’s tests of ground equipment included measuring and radio frequency, pneumatic systems, propellants, and the emergency detection system. The third week KSC conducted tests of the digital data acquisition system, camera control, and leak detection and purge panels. Although the contractors ran two-shift operations, shortcomings, particularly with the RCA 110 computer, forced a fourth week of individual stage tests. Integrated GETS tests followed during the week of 2 May.⁴⁶

In mid-April the Site Activation Board added another to its list of major problems: the installation of electrical and pneumatic lines on mobile launcher 1. Late deliveries and last-minute modifications from Huntsville and Houston threw the Pacific Crane and Rigging crews behind schedule. Despite 14-hour shifts, seven days a week, the crews made little headway. A “tiger team,” composed of members from KSC’s Engineering Division, the stage contractors, and Pacific Crane and Rigging, supervised the rush work. As the power-on date of 13 May approached, pipefitters abandoned their practice of turning over complete pneumatic systems to operations personnel. They began working line-by-line to meet specific 500-F milestones. As late as 3 August, 39 electrical cables and 232 pneumatic lines remained to be installed. Temporary fittings and work-arounds, however, prevented any major delay in test dates.⁴⁷

500-F rolled out from the assembly building on 25 May 1966, five years to the day after President Kennedy’s challenge. Despite the attendance of many Apollo program dignitaries, the Saturn vehicle stole the show. Like the Trojan horse of old, the first glimpse of the emerging Saturn vehicle was awesome. The crawler experienced little difficulty carrying its 5,700-metric-ton load to pad A. During portions of the trip, the transporter operated at full speed - 1.6 kilometers per hour. The Saturn vehicle reached the top of the pad at dusk and was secured two hours later. Understandably, the success was a joyful occasion for KSC and the entire Apollo team. The operation proved the soundness of the mobile concept.⁴⁸

Less than two weeks later, that concept received an unscheduled test under emergency conditions. In early June hurricane Alma skirted Florida’s east coast. Debus put the mobile concept through its paces. At 1:00 p.m. on 8 June, he ordered 500-F back to the assembly building. Within three hours, the launch team had disconnected the mobile launcher from its moorings. Wind gusts over 80 kilometers per hour spurred the efforts. The crawler began the return trip at 5:33, taking one hour to descend the 392-meter sloping ramp. Sheets of rain and 96-kilometer-per-hour gusts accompanied the crawler team on the straightaway as they urged their ponderous vehicle to its top speed. The crew reached the assembly building at 11:43 p.m. and had the mobile launcher secure on its mounts one hour later.⁴⁹

Lack of Oxygen Slows Apollo

For the first time KSC missed a major milestone in June when the Site Activation Board postponed the start of the wet test for 500-F. A series of events including hurricane Alma had slowed the Wyle Company’s cleaning of LOX lines. When Quality Division inspected Wyle’s cleaning of the crosscountry LOX lines, a powdery residue was found in the pipes. The cleaning compound, when mixed with the local water supply, had formed a precipitate. KSC prepared new specifications for the job, directing Wyle to flush the residue from the lines with an acid solution. LOX lines on the mobile launcher, contaminated during welding operations, also required recleaning. At a board meeting on 23 June, Scheller asked Roger Enlow to expedite contractual arrangements with Spellman Engineering, the company responsible for the LOX lines on the mobile launcher. Despite Enlow’s efforts, the cleaning lagged further behind schedule. On 1 July, Gruene and Scheller agreed that the wet tests could not start for another four weeks.⁵⁰

KSC’s program office may have underestimated the problem of keeping the cryogenic lines clean. In laying miles of pipe, workmen inevitably left debris behind. On one inspection half of a grinding wheel, broken pliers, and a glove were found in a LOX line. The use of invar for the inner pipe of the vacuum-jacketed lines further complicated matters. Invar, a steel alloy containing 36% nickel, had a very low coefficient of expansion, making it ideal for a cryogenic line; but it also rusted easily. During fabrication and installation, NASA inspectors had to watch for minute particles of dirt or moisture that might cause corrosion. When contamination was suspected, inspectors employed bore-sighting equipment to evaluate the potential corrosion. A decision did not come easily; inspectors spent a couple of weeks trying to determine if the LOX lines on the mobile launcher were rusting or were simply discolored.⁵¹

At the 7 July meeting of the Site Activation Board, Scheller announced the Apollo Program Office’s plan to delay the first Saturn V mission one month. Problems with the S-II test stage at the Mississippi Test Facility had prompted General Phillips’s decision. Although the directive appeared to lessen the urgency of the activation schedule, Scheller insisted that the board strive to meet the old 501 erection date. Marshall was constructing a dummy spacer as a temporary substitute for the S-II stage, and KSC would probably erect the AS-501 with the spacer to check the instrument unit.⁵²

During the delay caused in cleaning the LOX lines, the board scheduled the crawler’s first lift of the mobile service structure for 20 July. Mobile launcher 1, with 500-F aboard, was back at pad A where the crawler had transported it after hurricane Alma’s departure. The Bendix crawler crew spent two days in preliminary runs on the crawlerway and pad ramp and then carried the mobile service structure to the top of pad A for compatibility tests. Interest centered on the fit of the structure’s five clamshells around the Saturn. Launch officials also tested the service structure’s water deluge System.⁵³

The RP-1 fuel system was tested in mid-July by pumping 760,000 liters of kerosene from storage tanks to 500-F. As LOX line cleaning problems persisted, the Site Activation Board reduced the number of wet tests from twelve to eight and finally to five. Spellman Corporation completed its work on the mobile launcher LOX lines in early August, and KSC rescheduled the S-IC-F LOX loading for the 15th, when failure of both LOX replenishment pumps forced a cancellation. When the launch team tried again on the 19th, it ran into much bigger trouble.⁵⁴

Technicians in the launch control center began pressurizing the LOX storage vessel at 1:15 p.m. Simultaneously they opened the pneumatically operated pump suction valve, located in front of the 90 degrees elbow on the 46-centimeter suction line. This allowed a flow of LOX to cool down the 37,854-liter-per-minute pumps. What happened during the next two seconds kept several investigation boards busy for days. The gas caught in the 4.6 meters of piping between the storage vessel trap and the valve flowed out as the valve began to open (the valve was timed to open fully in ten seconds and had an eccentric pivot to aid in closing against upstream pressure). The rapid evacuation of the gas increased the velocity of the LOX flowing down from the sphere. The butterfly valve was only about 20% open when the LOX hit it with a water-hammer effect. As the liquid column backed up in the restricted passage, the pressure closed the valve disc back on its eccentric pivot. The corrugated bellows in front of the valve ruptured. A Boeing console operator in the control center secured the LOX system shortly after the accident, but he could only shut off the valves downstream from the rupture. A Boeing team at the storage vessel attempted to shut the block valve above the break manually, but the men were soon driven back by freezing LOX vapors billowing over the area. Within an hour, more than 2,700,000 liters of LOX poured out.^*

The sudden decompression caused the tank’s inner sphere to buckle. The outer shell retained its shape, but the collapse of a corrugated bellows, connected to the tank’s relief valve, indicated that a part of the inner sphere had caved inward. When technicians removed the perlite insulation between the two tanks, they found a depression in one quadrant of the inner tank. Some of the 9.5-millimeter stainless steel plates were bent 90 degrees from their normal curvature. While KSC officials were not sure how long repairs would take, NASA headquarters announced that the accident would delay the AS-501 launch by 45 days.⁵⁵

KSC faced two problems: repair of the LOX storage vessel and redesign of the system to prevent a recurrence. After draining the remaining 303,000 liters of LOX, the launch team used fans to circulate warm air through the inner tank. Following conferences with the manufacturer, KSC filled the tank with water. The stainless steel popped back into place at 0.4 kilograms per square centimeter (6 psi) and held its shape at higher pressures. KSC officials watched the operation over closed-circuit TV. While the water was draining, two engineers boarded a rubber raft to take a closer look from inside. Several days later, when engineers conducted a dye penetration test on the inner tank, no cracks were discovered. Technicians then replaced the perlite insulation between the inner and outer tanks. All damaged equipment had been replaced by 14 September. Meanwhile, KSC and Boeing engineers, with advice from consultant Peter C. Vander Arend, modified the system to fill the LOX line from the storage vessel to the pneumatically operated valve gradually, prior to chilling the pump. KSC replaced the flex hose where the break had occurred with hard pipe and substituted a pneumatic valve for the manual block valve.⁵⁶ The new circulation and precooling system was installed by mid-September, and the second loading of LOX into S- IC-F went off without incident on 20 September. The remaining tests followed in rapid succession. With all of the AS-501 stages at the Cape (the S-II spacer in place of the live second stage), KSC officials were anxious to get on with the real show. 500-F came down in mid-October, ending seven months of valuable service.⁵⁷

Management by Embarrassment

Although the Site Activation Board continued operations for another 20 months, it had made its major contribution to the Apollo program. The KSC team had successfully met its most difficult schedule, the activation of facilities for 500-F. Although some hectic days lay ahead, they involved the spacecraft rather than ground facilities. In July 1966, Rocco Petrone moved over from program management to launch operations. With him went much of the responsibility for site activation. From the beginning, program office representatives and launch operations personnel had argued over who should direct site activation. By the end of 500-F, the responsibility was shifting to Launch Operations and the Engineering Division.

Scheller, backed by Petrone, had won the opening rounds. At the time, the knowledge that the Site Activation Board had done its job was limited to KSC. Many outside observers did not believe that an American would be first on the moon. The Soviet Union had won the race to launch a multimanned spacecraft, sending Voskhod 1 aloft with three cosmonauts on 12 October 1964. The Russians conducted a second successful flight four days before the launch of America’s first manned Gemini. Gus Grissom and John Young’s three-orbit flight on 23 March 1965 went well, but earlier that month an Atlas-Centaur had exploded on the pad, causing over $2 million in damages. Aviation Week and Space Technology editor Robert Hotz commented after the successful Voskhod 2 flight: “Each Soviet manned space flight makes it clearer that the Russians are widening their lead over the U.S. in this vital area. It also makes it clear that the many billions the American people have poured willingly into our national space program for the purpose of wresting this leadership from the Soviets are not going to achieve that goal under present management.”⁵⁸

The activities described in this chapter helped render that judgment premature. One aspect of KSC management remains to be noted: the Site Activation Board developed a keen sense of competition. A “hit parade board,” prominently installed in the display room of the Activation Control Center, listed the ten most critical problems and the organization responsible for each activity. Unlike television’s Lucky Strike Hit Parade Board of older days, no one wanted this recognition. Civil servants, as well as contractors, were frequently embarrassed at the biweekly meetings. Hard feelings were inevitable, but the program’s goal helped pull organizations together. North American, a company that took as much criticism as anyone, reflected the spirit of fellowship in a going-away present to Rocco Petrone several years later. A common practice at board meetings (and also at later Apollo Launch Operations Committee meetings) had been to ask if anyone had constraints - situations that would hold up a schedule. North American officials presented Petrone a model of a Saturn V with the second stage missing. The sign on the space vehicle read, “Rocco, S-II is ready for roll except for one constraint.” The constraint: no S-II.⁵⁹