Long distance network link for TAx4 expansion

Long distance network link for TA×4 expansion a Institute for Cosmic Ray Research, University of Tokyo 5-1-5 Kashiwanoha, Kashiwa, Chiba 277-8582, Japan b High Energy Astrophysics and depertment of physics, University of Utah, 115 S 1400 E 201, Salt Lake City, Utah 84112-0830, USA c Graduate School of Science, Osaka City University 3-3-138 Sugimoto, Sumiyoshiku Osaka, 558-8585, Japan d Graduate School of Science and Technology ,Ritsumeikan University 1-1-1 Noji-higashi, Kusatsu, Shiga 525-8577 JAPAN e RIKEN, 2-1 Hirosawa, Wako, Saitama 351-0198, Japan E-mail: [email protected] As a high-energy expansion of existing Telescope Array (TA) experiment, TAx4 SDs have been deployed and operated from Apr.2019. Network communication for inter-site and each detector also have been expanded to longer distance. Communication towers are located in off-grid. Those are operated by solar panel and batteries. In this report we describe hardware configuration and installation as well as operation status after the installation. We measure reception signals at communication towers and each detector sites. The reception depends on condition of terrain. There are several models and tools those are used for predicting received power considering elevation model. For future ground experiment of air shower observation, it is useful if attenuation of signal strength and communication quality could be estimated quantitatively. In this report, we present observed signal strength and its comparison with available attenuation models also. 36th International Cosmic Ray Conference -ICRC2019July 24th - August 1st, 2019 Madison, WI, U.S.A. ∗ Speaker. c Copyright owned by the author(s) under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0). http://pos.sissa.it/ PoS(ICRC2019)372 T. Nonaka∗a ,R. Cadyb ,R. Fujiwarac ,E. Kidoa ,J.N. Matthewsb ,S. Ogioc ,T. Okudad , H. Sagawaa ,N. Sakakie ,S.B. Thomasb and K. Yada,a T. Nonaka TA×4 network 1. Introduction TA/TALE/TAx4 Array 30 km . KMCT , % # 0 %, , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % TAx4 North , % # 0 DMCT , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % MDFD " J ,% % ,% ,% , , ,% % ,% ,% ,% % , % , % ,,% % ,% ,% , % ,% % ,% , , % , , % , % ,% % , ,% , ,% % ,% , , % , ,% % , ,, % , % , % % ,% , % ,% % ,% , % % , % ,% % , % ,% ,% , % ,% ,% ,% , % , % , % , , % % , , % % , % , % , % , % , , % % , % , % , % , % % , , % ,, % , % , % , % , % , % , % , % , % # 0 SNCT # 0 # 0 MDCT , % , % 10 km TALE , % , % , % SKCT , % % , % , % , , % , % , % , % , % , % , % % , , % % , % , % , % , % , % , % , % , % % , , % , % , % , % , % , % , % , % , , % , % % , % , % , % , % , % , % , % , % , % , % , , % % , % , % , % , % , % , % , % , % , % , % , % , % , , % , % % , % , % , % , % , % , % , % , % , % , % , % , , % , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , CLF & , % % , % , % , % , % , % , % , % , % , % ,3 % , % , % , % , % , % , % , % , % , % , % , , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , , % , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , BRCT , % , % , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , , % BRFD , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , ,% , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , , % % , % , % , % , % , % ,% , % , % , % , % , % , % , % , % , % , % , , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , , % % , % , % , % , % , % , % , % , % , % , % , % , % , % , , % , % , % , % , % , % % , % , % , % , % , % , % , % , % , % , , % , % Delta, Utah TA % , , % LRCT LRFD % , , % , % , % , % , % , % , % , % # 0 " J TAx4 South 30 km 10 km # 0 20 30 , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % , % 2 % , % 2 % 2 % 2 % , % , % , % , % , % , % , % 2 % 2 % 2 % 2 % 2 % 2 % , % , % , % , % , % , % , % , % 2 % 2 % 2 % , % , % , % , % , % , % , % , % , % 2 % 2 % 2 % 2 % , % , % , % , % , % , % , % 2 % 2 % 2 % , % , % , % , % , % , % 2 % 2 % 2 % 2 % , % , % , % 2 % 2 % 2 % 2 % 2 % , % , % , % 2 % 2 % 2 % , % 2 % SRCT # 0 2 % , % , % SCCT 10 Figure 2: North array coverage of Line Of Sight from each tower. # 0 " J , % 5 ³ ² # 0 , % 0 CRC 40 50 Kilometers 22 Apr 2019 S. Thomas Dept. of Physics Univ. of Utah Figure 1: Layout of TAx4 experiment.Red dot represents deployed SDs for TA×4. Figure 3: South array coverage of Line Of Sight from each tower. 1 PoS(ICRC2019)372 TAx4 experiment is an expansion to increase statistics at high energy side of energy range of existing Telescope Array (TA) experiment[1, 2]. To obtain 4 times larger statistics of 5 years observation of TA experiment at energy of E ∼ 1019.7 eV, this expansion add ∼ 250 surface detectors (SD) at north side of existing array with 2.08 km spacing and another 250 SDs at south side as well. As well as TA experiment, TAx4 surface array is associated with fluorescence telescopes (FD) to enable hybrid observation which observe same air shower event with both types of detectors. Figure.1 shows project area includes TAx4, TA and TALE (TA Low energy Expansion )[3] experiment over wrapped with field of view of FDs for TAx4. A battery of FDs have been constructed at Middle Drum (39.473N, 113.994W) which observe sky at north robe and another battery have constructed at Black Rock Mesa (39.188N,112.712W) for south robe [4]. The project area is approximately 100 km from north to south and 50 km from west to east. Figure.1 shows detector those actually deployed before Apr.2019 and communication towers for SDs. T. Nonaka TA×4 network 2. Configuration of TAx4 north and south array 2.1 Distance and received signal strength(RSSI) at SD electronics The tower accommodates as data acquisition(DAQ) system for each sub-array. Each DAQ center which installed is consisted with transmitter (Air-AP1572EAC, Cisco make) which commercially used as Access point(AP) of WIFI system and a single board computer for data acquisition PC. The system currently employed is same setup with the one used in TALE[3] experiment. We use 5GHz band for tower-tower and 2.4GHz band for tower-detector. Before the tower site is decided, maximum distance can be covered with the AP have been made sure using detector located more than 18 km away. At the test operation, tower at TALE site (39.4331N,112.975W) and SDs at TA was used. At the test SDs have ∼ −85dBm of RSSI was able to be used for data acquisition. The error rate at the SDs for 24 hour operation was 0.27 %[5]. Assuming DAQ system try recovery communication at next cycle of every failure of communication, this error rate is affordable even with single retry. Estimated radio transmittance power for TAx4 SDs are obtained by two types of simulation such as "Radio Mobile"(RM) [6] and "link planner"(LP)[7] those consider obstruction by terrain (LP,RM) and reflection (RM). Figure.4 shows observed RSSI at surveyed detectors those located with good line of sight and model attenuation curve. Here the model attenuation curve is a formulae which added extra loss on free space propagation loss (fspl) model[8]. As it is shown in Figure.5, simulations suggests approximately ∼ 17dBm(LP) ∼ 12dBm(RM) of systematic discrepancy between expected power and evaluated at detector electronics. Here, 35 dBm of equivalent isotropic radiated power(EIRP) at transmitting point was assumed. This loss was taken into account at estimation of received power for TAx4 SDs to be sure. 2.2 Link lines and estimated loss Figure.6 shows all inter tower links in TA-TALE-TAx4 site. Link displayed in solid line used for data acquisition for TAx4. dashed line shows link used for general purpose also. Longest distance to be reached is 37 km between SNCT to DMCT. Considering power loss at propagation and required received power, the transmitter and receiver require 20 dBm of radio emitting power and relatively large receiving gain ∼ 30 dBi at 2 PoS(ICRC2019)372 Selection and adjustment of positions of towers and SDs have done based on field work. Tower sites and detector positions to request permission of land use were decided. There are three of tower sites at north array and south array respectively. Figure.2 shows the tower locations at north array with areas where line of sight from the towers cover. Figure.3 shows same at south array. Three towers at north array are at North edge of Smelter knoll hill (SNCT), Keg mountain (KMCT) and Desert Mountain (DMCT). At south array, one of the communication tower uses structure of Black Rock FD building (BRFD). There are other two towers at South cricket mountain (SCCT) and Sand ridge (SRCT). As it is shown, with the detectors and tower locations, it is possible to have a line of sight from all detectors to one or more of the three communication towers. Also, towers in same array are able to be linked each other from any other two towers via long distance network. T. Nonaka Number of SD RSSI, Received Power (dBm) TA×4 network RSSI and simulated receptions Good Line of Sight −20 Fit all Confirmed 6 RSSI read offset Link Planner - RSSI 5 Radio Movile - RSSI Only Confirmed −40 Radio Mobile Link Planner 4 −60 3 2 −100 1 −120 0 0 0 2 4 6 8 10 12 14 5 16 18 20 Distance (km) 10 15 20 25 30 35 Expect - Observed RSSI(dBm) Figure 5: Difference between simulated received Figure 4: Observed RSSI and simulations of reposer and observed RSSI. Evaluation were with deceived power. Error bars on data from TALE SDs tector where line of sight to TALE tower is very are for convenience, set as 3dBm for all data point. good. antenna. Figure.7 summarize estimated power loss between possible pair of link calculated by considering distance and terrain between two towers. Solid line in the figure indicates fspl for reference. 2.3 Capacity of inter tower link The function of communication tower is to enable continuous DAQ such as inter tower trigger, data transfer and misc operation. That require enough capacity of data flow through long distance link. Following items are the summary of required capacity to flow each type of data. • Daily data transfer from DAQ center PCs to Observatory in town. Expected and current data amount at each DAQ center is summarized in table.1 Table 1: Data amount transmitted through each line Link MDFD-SNCT SNCT-KMCT SNCT-DMCT SNCT-(Delta) BRFD-SCCT BRFD-SRCT BRFD-(Delta) Data KMCT,SNCT,DMCT KMCT, DMCT, TAx4(North), TALE, TASD, Misc for FD SCCT, SRCT, TAx4(South), TASD, Misc for FDs 3 Data amount(MB/h) 26MB 7MB 4MB 69MB=(26+18+25)MB +FD 8MB(expect) 8MB 87MB=(25+62)MB +FD PoS(ICRC2019)372 −80 T. Nonaka Latitude TA×4 network KMCT DMCT New Com Towers 39.7 TAx4 network TAx4 SD TA network TA SD 39.6 TALE SD FD stations Old com towers 39.5 Central Laser Observatory SNCT 39.4 39.3 BRFD 39.1 39 SRCT SCCT 38.9 −113.2 −113 −112.8 −112.6 −112.4 Longitude Figure 7: Simulated power loss between each towers. Calculation consider obstruction by terFigure 6: Entire long distance link between sites in TA- rain. TALE-TAx4 experiment. • Sending trigger information between tower. There are two types of data in the inter-tower trigger. One is the time information of the signal from the detector in the border area of each sub array, and it is used to form an air shower trigger. The another is to send Air shower trigger information between towers. The amount of data sent to other towers is estimated based on the maximum SD number of each tower. The maximum amount of data sent to any line is estimated to be approximately 60kB/sec. ( = 3(tower) × 100(SD) × 0.2(kB/sec·SD) ) 1 • Control of DAQ host and access point. Usual operation at DAQ PC at tower is done via text terminal. Except for occasional maintenance for instruments those have web based interface for setting, capacity required for control DAQ PC and access point is negligible. 3. Installation and operation status of towers The tower consists of a tower mast and a power supply module. A total of six towers were built in March 2019. Four tower masts were assembled in Holden (south east of Delta City). Fourteen power modules were manufactured at observatory in Delta City. Each power module consists of solar panels and batteries, and the container for storing them also accommodates a single board PC and network equipments for data acquisition. They were transported from the Delta city to near the location of the tower by trailers and lifted from there by helicopters. For the South Cricket Tower, the tower site is located at the top 1 The actual system currently under development sends information from two towers to one of the three towers at center. Therefore, the amount of trigger information sent to one transmission path will be much smaller than the above-mentioned estimated maximum value. 4 PoS(ICRC2019)372 39.2 T. Nonaka TA×4 network of the hill, so instead of a huge tower mast, we installed an 8 m long high-tension pole. The pole is equipped with a winch system to be able to be pulled up and down, so that installation and maintenance of network devices can be done without climbing. 3.1 Tower hardware Figure 8: One of the constructed tower (SNCT). 12m mast and three power modules on the Figure 9: One of the constructed tower (SCCT). ground can be seen. Total power consumption This tower is located on the top of a hill about 60 of this tower is 89W/h m high above surrounding ground level. this DAQ system, the estimated power consumption is up to 25W. The power consumed by each antenna for long distance network is 17W. The power module contain seven batteries. The output of solar panel and battery are combined between power modules to be connected to a charge controller (TS-MPPT-60, manufactured by MorningStar). Each tower has two or three power modules, depending on power consumption. Table2 shows the total power consumption of each tower and the number of modules. Table 2: Number of power sources and estimated power consumption at each communication site. Site name Module Power(W) KMCT 2 55 DMCT 2 55 SNCT 3 89 SCCT 2 55 BFCT 2 77 SRCT 2 55 Table 3: Frequency and transmitting power (dBm) at installed long distance link Link MDFD-SNCT SNCT-KMCT SNCT-DMCT BRFD-SCCT BRFD-SRCT 5 (km) 13 27 37 32 30 Frequency (MHz) 5700 5240 5800 5240 5800 EIRP (dBm) 30 49 50 49 56 Capacity (Mbps) ∼34.3 ∼38.3 ∼37.2 ∼57.9 ∼38.3 PoS(ICRC2019)372 Figure.8 shows a tower mast built for the communications tower. The tower (KT5M-2AE, Creative Design make) is 12 meters high and has two elevator ports for mounting network devices such as antennas. The tower is supported by 3 pair of guy wires connected to anchor point on the ground rock. It’s vertical is made sure approximately within 0.5 deg at installation and small adjustment can be done by tuning turn buckles at lower part of guy wires. Three solar panel-battery unit each equipped with three solar panels (DS-135, Dasol make) enclose network instruments and batteries(DCS100 C&D technologies make) are shown in same photo. In T. Nonaka TA×4 network 3.2 Inter tower link Data link between tower to tower is achieved by high power point-to-point network modules (AF-5x, Ubiquiti network make) which have maximum 26 dBm of Transmitting power equipped with 30 dBi gain antenna. Condition of each link is summarized in Table.3. 3.3 Link stability 10 SN-MD 8 loss rate(%) loss rate(%) 10 6 4 2 SN-DM 8 6 4 2 0 4 2 SN-SC 8 6 4 2 0 09 Thu 10 loss rate(%) 6 0 10 loss rate(%) loss rate(%) 0 10 SN-SR 8 SN-KM 8 16 Thu 23 Thu 30 06 Thu Thu Date 13 Thu 20 Thu 27 Thu 6 4 Site name SN-KMCT SN-DMCT SN-MDFD BF-SRCT BF-SCCT (%) < 0.01 < 0.34 < 0.01 < 0.30 < 0.04 2 Figure 11: Average packet loss rate. 0 09 Thu 16 Thu 23 Thu 30 06 Thu Thu Date 13 Thu 20 Thu 27 Thu Figure 10: Packet loss rate(%) at each line. The loss includes misc maintenance also. 4. Summary and future TAx4 SDs and communication towers were deployed and constructed. Major construction have finished at end of March 2019. For this project, network communication for inter-site and each detector have been expanded to longer distance. In this report, preparation of hardware and operation status after the installation was described. Locations of tower and detector have been chosen to have coverage of line of sight from all detector. Data acquisition at longer distance was tested to make sure that the larger array size is practically achievable. Most of all deployed detectors for TAx4 are communicated from tower fine. Observed RSSI tend to be larger than expectation based on measurement at TALE site about same amount we estimated conservatively. We would summarize observed RSSI data. Now all SDs are under commissioning to finalize preparation for scientific data acquisition. Still this is only few month of operation, however, communication towers and inter site link is working stably with enough capacity for later trigger scheme. 5. Acknowledgements The Telescope Array experiment is supported by the Japan Society for the Promotion of Science(JSPS) through Grants-in-Aid for Priority Area 431, for Specially Promoted Research 6 PoS(ICRC2019)372 After installation of tower components and network system, those towers have been operated for commissioning detectors and DAQ scheme. Data transfer is regularly done up to observatory. Figure.10 and Table.11 shows stability of each new line. There still room to improve alignment at one line but enough stability to do operation and sending information to other towers. T. Nonaka TA×4 network References [1] H.Kawai et. al. Telescope array experiment. Nuclear Physics B - Proceedings Supplements, 175176(0):221 – 226, 2008. Proceedings of the {XIV} International Symposium on Very High Energy Cosmic Ray Interactions. [2] T. Abu-Zayyad et al. The surface detector array of the telescope array experiment. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 689(0):87 – 97, 2012. [3] S Ogio. Development of the tale surface detector array. Proc. 34thICRC, page 637, 08 2016. [4] E Kido. The tax4 experiment. (This conference), page TBD, 07 2019. [5] R Sahara. Development and operation of a new data acquisition system for the tale surface detector array. Master’s thesis, 03 2018. [6] R Coudé. Radio mobile website, 1997. [7] Cambium Networks. Linkplanner-quickly design network, 2019. [8] H. T. Friis. A note on a simple transmission formula. Proc. IRE., 34:254–256, 1946. 7 PoS(ICRC2019)372 JP21000002, for Scientific Research (S) JP19104006, for Specially Promoted Research JP15H05693, for Scientific Research (S) JP15H05741, for Science Research (A) JP18H03705 and for Young Scientists (A) JPH26707011; by the joint research program of the Institute for Cosmic Ray Research (ICRR), The University of Tokyo; by the U.S. National Science Foundation awards PHY-0601915, PHY-1404495, PHY-1404502, and PHY-1607727; by the National Research Foundation of Korea (2016R1A2B4014967, 2016R1A5A1013277, 2017K1A4A3015188, 2017R1A2A1A05071429) ; by the Russian Academy of Sciences, RFBR grant 16-02-00962a (INR), IISN project No. 4.4502.13, and Belgian Science Policy under IUAP VII/37 (ULB). The foundations of Dr. Ezekiel R. and Edna Wattis Dumke, Willard L. Eccles, and George S. and Dolores Doré Eccles all helped with generous donations. The State of Utah supported the project through its Economic Development Board, and the University of Utah through the Office of the Vice President for Research. The experimental site became available through the cooperation of the Utah School and Institutional Trust Lands Administration (SITLA), U.S. Bureau of Land Management (BLM), and the U.S. Air Force. We appreciate the assistance of the State of Utah and Fillmore offices of the BLM in crafting the Plan of Development for the site. Patrick Shea assisted the collaboration with valuable advice on a variety of topics. The people and the officials of Millard County, Utah have been a source of steadfast and warm support for our work which we greatly appreciate. We are indebted to the Millard County Road Department for their efforts to maintain and clear the roads which get us to our sites. We gratefully acknowledge the contribution from the technical staffs of our home institutions. An allocation of computer time from the Center for High Performance Computing at the University of Utah is gratefully acknowledged.