- Recovery Operations Control Room (ROCR)Because the recovery assets were almost all from the DOD, i.e., USN or USAF, they were managed by NASA through inter-agency agreements. The Recovery Operations Control Room included people from the services charged with recovery of the astronauts and spacecraft. The large room #327 had large screen displays supported by a back room with projection equipment that displayed the locations of all the recovery assets for both planned and emergency situations. It was located to the right of the entrance to the MOCR 2 and behind a glass wall. It could not be seen by the Visitor Viewing Room but could be seen from the DOD Manager’s position in the MOCR. As such it will not be restored due to cost. Only a large photo of the room will be attached to the glass window.
It was the responsibility of the Landing and Recovery Division (LRD), which was headed by Jerry Hammack. For the Apollo lunar missions, with only one or two exceptions, the ROCR NASA personnel came from this Division. There were five Branches in the LRD as follows:
- Recovery Operations Branch Chief: Dr. Donald E. Stullken.
- Planning and Control Branch Chief: Harold Granger.
- Recovery Electronics Branch Chief: William R. Chase.
- Operations Test Branch Chief: Weldon B. “Gus” McCown.
- Recovery Systems Branch Chief: John C. Stonesifer.
The ROCR was similar in layout to the MOCR. It was equipped with smaller forms of the large screen group displays. On the left was a Projection Plotboard to display the world map and ground tracks. In the center were two screens that were supported by vu-graph machines on earlier flights, then projectors on later flights. The projectors were used in conjunction with the overhead cameras. The central screens displayed the status of recovery forces and mission events. On the right was an Eidophor display that was used to present console systems displays or video feeds, such as real time KSC or MCC live links or downlinks from the crew. Above these screens were time displays to show GMT, Mission Elapsed Time, Flight Events, countdown clocks, etc.
Along the left wall was the P-Tube station. From the right of that was a long plot table with map files beneath it. The table extended for the length of the wall and was used for plotting and selecting target points.
In the back of the room was a smaller room (two rooms on the later missions) with the DOD Assistant for Communications, encryption equipment, and a TTY machine.
The ROCR had approximately 15 console positions for NASA and DOD people.
- Simulation Control RoomThis room #328 included the trainers of the flight control team who conducted launch, orbit, descent, landing, and reentry simulations. Each vehicle had a console position; the Command and Service Module (CSM), the Lunar Module (LM) and the Saturn Launch Vehicle (SLV) as well as the communications and tracking network. These operators could simulate various types of failures in order to assess the flight control teams’ reactions. This required computer simulation software that would drive the flight controllers’ displays. Often, new operational procedures and mission rules were written as a result of these simulations. Even the astronauts’ checklist and procedures could be rewritten.
The Simulation Team faced the MOCR 2 through a glass wall so they could see the flight controllers’ reactions to the failures. The second floor Simulation Control Room included more consoles for training and the equipment to drive the simulations. The following positions included:
- Simulation Supervisor
- Lunar Module/Saturn Launch Vehicle
- Tracking and Trajectory
- Command and Service Module
Because it is visible from the MOCR and partially from the Visitor Viewing Area, this room will be partially restored to the Apollo Lunar Landing era if funds become available. The room will be cleaned, the consoles restored and put into the proper locations. Other equipment will be removed. The wall-mounted P-tube station will remain. The room should appear from the MOCR 2 as it was in the Apollo era complete with lighting and consoles lights/event indicators. As currently planned, none of the SSRs are being restored so this room could be representative of the others.
- Spacecraft Analysis (SPAN) RoomThe SPAN is the Apollo Spacecraft Program Office (ASPO) Manager's official interface with the Flight Operations Directorate (FOD) and the Science & Applications Directorate (S&AD). The ASPO will have the capability to provide detailed analysis and answers to questions asked by the FOD prior to and during real-time flight operations. This includes questions relating to experiment hardware, spacecraft operations and mission requirements.The SPAN Room is located on the third floor in Room 312A adjacent to the MOCR 2. It is staffed by the following ASPO personnel:
- SPAN Operations Manager (the senior ASPO representative)
- Mission Planning and Analysis Division (MPAD) Senior Representative
- Marshall Space Flight Center Lunar Roving Vehicle Senior Rep.
- Mission Staff Engineer – Detailed Flight Objectives
- Contractor Representative (NR/GAC/MIT)
- Crew Systems Division( CSD) Senior Representative
- Log Manager-Action items
- Messenger-Hand carry actions and responses to other rooms
- Operations Manager.
The SPAN is also staffed by the following FOD personnel:
- FOD Senior Representative
- Flight Control Division (FCD) Senior CSM Representative
- FCD Senior LM Representative
- FCD Senior Flight Dynamics Representative
The SPAN is also staffed by the S&AD Chief, Science Mission Support Division to coordinate with personnel in two other rooms with science teams for both orbital and lunar surface experiments such as the Scientific Instrumentation Module (SIM) and the Apollo Lunar Science Equipment Package (ALSEP).
- Mission Evaluation Room (MER)The mission evaluation team provided engineering and technical support to the Mission Control Center (MCC). This multi-disciplinary team provided detailed analysis of spacecraft problems to the Apollo Spacecraft Program Manager’s representative who was stationed in the Spacecraft Analysis (SPAN) room.A group of specialists was assigned to each engineering discipline necessary for a mission evaluation. The evaluation team, working in the Mission Evaluation Room (MER) in Building 45, provided support 24/7 during the mission, using three shifts of personnel organized under individual shift team leaders. All teams reported to a NASA shift manager (also called the team leader) who was responsible to the evaluation team manager, both of whom were members of ASPO. The specific disciplines represented for each spacecraft’s systems were: telecommunications, crew systems, electronic systems, propulsion and power, guidance and control, structures and mechanics, and thermal control.In addition, there were specialists for the Apollo Lunar Surface Experiments Package (ALSEP); safety, reliability, and quality assurance and flight crew training. Each shift team of specialists included NASA employees and contractors. They worked as a unit under a NASA team leader who directed team efforts, resolved problems, scheduled evaluation tasks to meet time constraints, coordinated with other team leaders to ensure that resolutions or recommended actions did not jeopardize other systems reviewed and approved the systems evaluations and wrote the daily summary reports.Corresponding teams of specialists were located in a mission support room at each of the two spacecraft contractor facilities; North American Rockwell and Grumman Aircraft. The efforts of each support team were coordinated through a contractor senior engineering manager who was assigned to the mission evaluation team and worked directly with the shift manager and the evaluation team manager.
Over a hundred people supported the MER during the Apollo missions.
- Spaceflight Meteorological RoomThis room was responsible to the MOCR and other SSRs and support rooms for meteorological and space radiation information. It supported the Recovery Support Room for weather analysis for the launch and recovery phases as well as the deep space radiation environment which had potential effects on the crew and therefore was of great interest to the MOCR flight surgeon and the Life Sciences SSR. It was generally a 24/7 three shift operation with the following console positions:
- Meteorologist (NASA, Weather Bureau and ESSA representative)
- DoD Defense Meteorological Service Officer
- Space Environment Console (SEC)- instrument specialist
- SEC- radiation dose specialist
- SEC- solar physics specialist
- SEC-Team Leader
- SEC- Aeromed
- SEC- Officer
- Goddard Communications Representative
This room was staffed with up to 15 people depending on the mission phase.
- Visitor Viewing Room (VVR)When first built in 1964, the VVR (often called the VIP room) directly behind the MOCR 2 was 65 feet long and 26 feet wide and accommodated 74 visitors in five tiered rows of theatre-style seats. The room was often over full and many stood and often sat on the floor. They looked down on the flight controllers and the large screen displays through a wall of glass. There is a shelf in front of the front row with head set jacks so those in the front could listen to the flight directors and other conference loops. These were usually reserved for astronaut families or dignitaries. A standing desk was provided behind the top row. In the two corners were two small (7’x8’) Communications Booths and in the other two corners were four telephone booths.Decades after Apollo 11, modifications were made to the area to accommodate tours including extending the elevator to the third floor and providing better access to the room including ADA-compliant access. Modern flat-screen monitors were added as well as murals and photos on the walls. If funds become available, this room will be restored and be the central focus for those wanting to see the Apollo Mission Control Center as it was during the epic flights to the moon.
- Auxiliary Computer Room (ACR)This facility was located in the Administrative Wing of building 30 in rooms 3050, 3051 and 3052. It was staffed by several Branches of the MPAD including: Flight Analysis, Rendezvous Analysis, Orbit Design and Math Physics Branches and provided support for trajectory aspects of flight control. During Gemini and early Apollo it conducted mission-critical computations for unexpected trajectory computations, and engineering evaluations of trajectory problems that developed during the missions. The facility included an IBM-7094 and 1401 computers staffed by the Flight Analysis Branch of MPAD and support contractor, Lockheed. The facility functioned as a full scale mission support activity until after the Apollo 11 mission when it gradually reverted to an off-line and on-call service. This downgrading reflected the increased capability and flexibility of the RTCC for the lunar landing missions. Once verified, these new capabilities eliminated the need for redundant computations.The Auxiliary Computer Room (ACR), also called the Real Time Auxiliary Computer Facility (RTACF) in some documentation, was an extension of the original computing facility used during the Gemini Program. The computer equipment was managed and operated by the MSC Central Data Facility in Building 12.Also in the room were desks with communications loops to monitor the mission and to communicate with the Trajectory Support Chief in the Flight Dynamics SSR, overhead TV monitors of the MOCR displays and photo tables to send data to the Flight Dynamics SSR or the MOCR. Data was often hand carried. The ACR was staffed with up to 45 people over a three shift operation.
During the initial Gemini Program, the MPAD recognized the need to augment the RTCC software which was frozen several months prior to flight. The Gemini flight schedule was very tight and the flight requirements changed rapidly. The ACR was designed to utilize independent programs for mission and simulation support. The software and planning tools were later integrated into the RTCC software for subsequent missions. The ACR support for Gemini began in a very limited way for GT-3 for retro fire and reentry computations. By GT-4, the ACR was manned 24/7 with three shifts as the Houston MCC was prime for that mission.
During Apollo, the ACR facility was expanded with mission experience to include support of all areas of flight control and included both mission and mission-simulation support. The facility functioned as a full scale mission support activity until after Apollo 12 when the facility gradually reverted to an offline, on-call operation since the RTCC capability and flexibility was increased for the lunar landing missions. These new capabilities were sufficiently verified in the RTCC so the need for redundant computations was eventually discontinued.
While the term “real-time” was sometimes used in its formal name, it was rarely used in real-time as the RTCC supported that function; it was, however, used to support the ongoing missions. The ACR would be used to provide mission-critical computations for unexpected trajectory computations and engineering evaluations of trajectory problems that developed during the missions.
The positions were staffed with three shifts and included the following positions:
- ACR Chief (Primary interface with the Flight Dynamics SSR)
- Trajectory Analysts and Mission Phase Specialists
- Program Consultants
- Computer Run Coordinators
- Engineering Aides and Key Punch Operator
- Real Time Computer Complex (RTCC)When the MCC-H was designed in the early 1960’s, there were only three mission operational functional systems: the Communications, Command and Telemetry Systems (CCATS), the Display/Control System and the Data Processing System/Real Time Computer Complex (RTCC). Room #112 of the first floor of Building 30 housed the RTCC. During Gemini, it included five IBM 7094s. Three of which included the Mission Operational Computer (MOC), the Dynamic Standby Computer (DSC) and the Simulation Operations Computer. The two other machines included the Ground System Simulation Computer and a standby for future software development. The Ground System Simulator functioned like the tracking network, telemetry network, and related ground-based parts of mission control to facilitate the testing of other software.
The RTCC system performed the necessary operational, real time computation tasks, and also generated data in support of the system testing and flight controller training and simulation activity. Each of the five computing systems in the RTCC consisted of the following: the Real Time Computer Subsystem, the Computer Control Subsystem and the Auxiliary Data Processing Subsystem. Over the years, these systems were upgraded as the technology advanced. The following sections describe the system as it was introduced for the early Gemini missions.
Basically, the system processed all the tracking and spacecraft telemetry data from the remote sites, processed all the commands from the control center to the spacecraft, and computed the current trajectory of the spacecraft from launch to orbit and all orbital maneuvering through to entry; probably the most complex of all the computer’s functions. All the while, the RTCC was sending information in real time to all the displays and event lights used by the flight controllers on their consoles in the MOCRs, SSRs, and Simulation area.
The heart of the system was a processor which ran at a frequency of 1 million instructions per second (MIP). This equated to about 500,000 logical operations, 250,000 additions, 100,000 multiplications, or 62,500 divisions per second. The data were displayed using black-and-green digital TV monitors and binary event indicators. During development and software testing, memory dumps were often taken and written out on high speed printers. Each computer occupied an area of 160 square feet of floor. Incoming tracking and telemetry data, and outgoing command data, were interfaced to the RTCC through the two UNIVAC 490s of the Communications, Command and Telemetry System (CCATS).
After the final Gemini mission in November 1966 and the Apollo 1 accident in January 1967, MSC and IBM took advantage of the stand down to upgrade the RTCC. The 7094s were replaced by five IBM 360-75Js with the new IBM 360 operating system. These were the computers that supported all of the manned Apollo missions. Again, the new software was a major step up. There were more mission phases, more vehicles capable of greatly increased telemetry downlinks and command uplinks, new trajectory algorithms, onboard navigation sources, more MOCR flight controllers, and the use of both the Space Tracking Data Network (STADAN) and the Deep Space Networks (DSN).
After Apollo, the RTCC was once again upgraded to the IBM 370 to support the Space Shuttle program, which was an even more demanding function. And after the loss of Challenger in 1986, NASA used the stand down to upgrade the RTCC to the IBM 3083s before the resumption of flights in 1988. But that configuration was short lived. In September 1990, IBM announced its new family of Enterprise Systems Architecture (ESA)/390 operating systems. This was a new “connection architecture” with many functional enhancements.
But the computing world was moving away from large mainframe computers to distributed architectures. For the International Space Station (ISS) there would be new control rooms using networked UNIX workstations, with high resolution color displays and laser printers.
- Communications, Command and Telemetry SystemsThe CCATS (pronounced “Sea Cats”) was located on the first floor of Building 30 adjacent to the RTCC. Every voice communication, telemetry signal, tracking data and spacecraft command went through the CCATS. And, every signal that passed to each spacecraft, astronaut or experiment, whether in orbit or on the Moon, went through the CCATS. That included the biomedical, television, radar signals and systems information. The CCATS connected the Mission Control Center to the Space Tracking and Data Acquisition Network (STADAN), the Manned Space Flight Network (MSFN), the NASA Communications (NASCOM) Network, the Apollo Launch Data System (ALDS), and the Deep Space Network (DSN).
For Apollo, the Manned Space Flight Network to which the CCATS was connected comprised 14 remote stations, each with two computers, as well as computers at other sites. There were a total of 39 Univac 642B computer systems (to replace the 1218s used in Gemini) around the world to relay telemetry and command information between Houston and the Apollo spacecraft.
The CCATS was also connected to five Apollo Instrumentation Ships (AIS) that provided crucial tracking, telemetry, command, and voice communications. Three were insertion & injection ships and two were reentry ships. There were also eight EC-135A Apollo Range Instrumentation Aircraft (ARIA) for voice communications and telemetry recording. Only two satellites were available for relaying data during Apollo, one Intelsat over the Atlantic and one over the mid-Pacific. Connected stations could relay to Goddard and Houston. The Tracking and Data Relay Satellite System (TDRSS) had not yet been built.
The CCATS real time system decommutated and distributed incoming data into the RTCC, console displays, and assorted monitoring devices. It formatted and then issued command data to Goddard for distribution to the desired remote sites. All digital data that was routed through CCATS was processed by one of the three UNIVAC 494s that were adjacent to the CCATS operators. Normally one computer served as the Mission Operations Computer (MOC) and another as the Dynamic Standby Computer (DSC). In the event of the MOC failing, an operator would manually switch over to the DSC. The third computer in this facility was used for checkout, debugging, and local operations. In effect, the 39 remote site UNIVAC 642B computers talked to the three UNIVAC 494 computers in the CCATS, which talked to five IBM computers in the RTCC, to give the fight controllers in the MOCR what they needed to carry out mission operationThe primary interface between the CCATS and the MOCR passed through the Network Controller and his Network Support Team. The CCATS room operated on three or four shifts, depending on the mission, with each shift having about 20 people.
- Display Projection RoomsWhen someone walks into the Visitor Viewing Area, the first things they see are the large screen displays that dominate the far wall. These and all the other displays on the consoles and over the projection screens give the flight controllers the data they need to perform their various functions. The group displays provide an immediate visual representation of the status of the mission from different perspectives.
When the MCC-H was designed, the flight controller requirements included many different types of displays. Almost all of these displays were controlled to some degree by the RTCC and selected by a flight controller as appropriate. The Display/Control System provided output to the entire building and even in the other wings of the building. To that extent, no single room house the system; it was a distributed system included dynamic and reference displays, and made use of plotting, television, and digital methods to present the data. For example, the dynamic displays would include real time telemetry from the spacecraft systems and crew biomedical data. Reference data included such information as mission rules, operational procedures, nominal sequences or historical performance data. The system included ways of formatting such data for the flight controllers. Some of the displays would go directly to the flight controllers’ consoles, while other data would go to the group displays.
The original display technology included a Projection Plotter Display that comprised a set of seven projectors: one background, two spotting, and four scribers. The background projector displayed the world map from a square slide that was 1 inch square. The spotting projectors imposed symbols to represent the spacecraft or a target on the map, then relocated the symbols to match trajectory data supplied by the RTCC. The computer also controlled the scribing projectors that used diamond-tipped styli to scratch alphanumeric character or an X-Y plot through the metalized coating on a glass slide. A 2500 watt xenon lamp projected the slide onto the screen. There was also an Eidophor projection system which is also now obsolete. This produced the images most people are familiar with in videos or photos of the control center. This complex optical technology was part of the Group Display Subsystem that provided varying degrees of information to all the control areas: the MOCRs, the Staff Support Rooms and the Recovery Control Room, as well as to the RTCC on the first floor and the Mission Briefing and Observation Auditorium and the Auxiliary Computer Room which were located in the Administration Wing.
There were four 10 x 10 foot projections screens and one 10 x 20 foot screen. They could each use either or both systems. The Eidophor used folded optics so that the optical throw distance required for quality projections could be achieved in a shorter space. Nevertheless, the rear projection room was very large (36 feet wide, 65 feet long, and 15 feet high) with its walls, ceilings and floors all painted black; no wonder it was known as the “Bat Cave.” The Eidophor was replaced in the early 1980s with GE light-valve type projectors; they, in turn, were replaced in 1990.
In the late 1980s, during the Shuttle era, it was decided to further upgrade the display system to the Hughes HDP-6000B projector that used liquid crystal displays. After fabrication and testing, the new projectors were installed in both MOCRs during 1990. Each installation consisted of the projector, new screen, projector lift table, workstation/graphics processor, application software, maintenance monitors and maintenance documentation.
Although the room behind the large screen in the third floor Apollo Mission Control Center will not be visible to visitors, it is considered an extension of the MOCR and be utilized in the restoration to some degree. In fact, there is another area even behind the projection area (called the “Bat Cave”) that is even larger and was originally filled with display control terminal equipment. Other modern means of projection will be used for the restoration to give the viewer/tourist a high fidelity representation of Apollo displays.
- Other Equipment AreasA building of the size and complexity of the MCC-H includes many rooms that are used for mechanical equipment, heating and air conditioning, testing, facility maintenance and telephone and voice communications and storage. There were also restrooms on either end of each floor. Some of the specific equipment on the first floor included: Voice, Teletype, Communications, Common Carrier Equipment, Distribution Frames, Tape Storage and Local Telephone Maintenance Rooms. In addition, there were various mechanical equipment rooms on the other two floors. Adjacent to Building 30 was an Emergency Power Building #48 of 10,500 square feet that contained all the standby equipment for power generation, air-conditioning, and interim and backup heating facilities.