A Ubiquitous Home Network System for Managing Environment-Information Sensors using Image Processing

With the recent rapid development of semiconductor and communication technologies, a wide range of home appliances has become digitized. Instead of existing independently, these digital home appliances can be interconnected for data communication, offering the potential of providing better services for users. A home network becomes the center of such development. A number of different types of home networks have been proposed. The wireless home network is very attractive as it obviates the problem of home wiring and allows for convenient movement. A wireless home network is also very suitable for a home network that gathers environmental information and controls all home and outdoor equipment. In this study, the ZigBee protocols were applied and implemented as part of home automation. The current ubiquitous sensor network is a wireless network where sensor nodes are of compact size, low cost, and low power. These nodes are randomly arranged in a wide area to gather environment-related information. Though the form of a USN (Ubiquitous Sensor Network) is the same as the form of an ad hoc existing wireless network, it has technically different characteristics from a wireless ad hoc network in terms of network scale, energy efficiency, limited node resources, etc. While a wireless ad hoc network is built for the purpose of providing communication services between mobile nodes even without the presence of a communications infra-structure, the USN is built for gathering environment-related information. Thus, the factors that are important to it are the subjects of sensing and the mobility of the collection points (the sink nodes), rather than the mobility of the nodes, because such factors require frequent flooding or rerouting to gather sensing information, which results in the consumption of significant amounts of energy from the USN. It is very important to take these characteristics into account in designing the network protocol of a USN. However, most existing studies have proposed to gather a range of home sensing information without allowing for sensing or for the mobility of the sinks [ 1, 2]. The network layer of the USN requires a network protocol that considers the subjects of the sensing and the mobility of the sinks while minimizing energy consumption. A USN having this feature would be suited specifically to obtaining and utilizing information around the home. Sensors around the home can obtain information on various environmental changes, and this information can be distributed and shared by utilizing a wireless sensor network composed of small-sized sensor nodes where sensing, data processing, and communication are enabled. Data, measured from the sensor nodes, is transferred to a server with the function of gathering, processing, and delivering all the data to the user. This process is designed with a consideration of sensor nodes with an energy limit. In this study, we implemented a sensor network for ubiquitous use based on image processing in order to minimize energy consumption. This network allowed for the characteristics of an existing network protocol. In Section 2, we discuss related work. In Section 3, we describe our system model, and in Section 4, we describe the implementation of this proposed home network system. Finally, in Section 5, we offer our conclusions.A number of projects in ubiquitous computing provided motivation for our work. Examples include the Intelligent Room (MIT), the Interactive Workspaces Project (Stanford University), the Aura Project (CMU), and the Easy Living Project (Microsoft). These projects have all been successful in creating systems that interact effectively with the user and the environment, but their applicability to other projects is limited because the systems are proprietary and are based primarily on nonstandard protocols. Furthermore, the systems are generally limited to a single organization or building. The work we describe here is built on open protocol standards like SIP, SLP [ 1], Bluetooth technology, and the ongoing efforts from the Internet Engineering Task Force (IETF). Active Badge is a system used to determine the location of individuals in an office environment. Each individual wears a small badge that periodically sends out a pulse-width modulated infrared (IR) signal bearing the identity of the person wearing it. Each building is equipped with many IR receivers connected to a central server that detect these signals. RADAR uses RF signal strength and a two-phase approach to estimate the location of a mobile target. In the initial setup phase, signal properties at well-known reference points are gathered and stored. During the usage phase, mobile devices measure the strength of beacon signals received from 802.11 access points, and these strengths are compared to the pre-measured signal strengths (of the reference-points) to estimate the mobile nodes’ locations. MoteTrack, which builds on RADAR, uses Crossbow Mica2 sensors instead of the 802.11 wireless specification and offers a location accuracy of two to three meters indoors. Mote-Track is intended for emergency-response applications and is used to provide rescuers with a heads-up display that tracks their locations in a building and shows safe exit routes. The mobile nodes in MoteTrack are assumed to be connected to a computer that drives a display unit. Like RADAR, MoteTrack uses a two-phase approach, and the location of a mobile node is taken to be the centroid of the most closely matched reference points. Homeplug AV (HPAV) is designed to provide the best connectivity at the highest QoS for the home networking technologies designed for home audio/video applications. HPAV consists of HPAV PHY, HPAV MAC, the Convergence Layer (CL), the Connection Manager (CM), and the Central Coordinator (CCo) [1, 2, 3, 4]. The research described in [1] and [3] provide an access control method, and the research in [3] proposes a framework based on an OSGi (Open Service Gateway initiative) service platform. OSGi provides an open service platform for a home network [4]. It defines a lightweight framework used for delivering and executing a service-oriented application. Other researchers [2] have proposed a SSL (Secure Socket Layer) Component for an enterprise environment. Although the efforts described in [3] and in [1] provide compatibility and economical efficiency to users and managers, there are some limitations to this work related to information security.
   Existing home network systems are intended mostly to
gather and manage home information, but a true home network will be able to gather and manage all environment-related information from both the home and outdoors. Home information generally involves little movement of information and focuses more on data service as compared with outdoor information, which tends to involve more movement of information. Because of this movement and because outdoor information includes a greater range of information types, outdoor information requires a more efficient way to gather data. With this in mind, we designed our network as shown in Fig. 1. In this home network system, both the home network management server and the database management server manage the home and outdoors. For this study, we gave only small consideration to the home communication network; instead we focused on the outdoor communication network for the sensor network. We chose to use a server with a PXA 255 embedded system for the outdoor communication network. This system includes a built-in sensor to gather information from the sensor gathering external environmental information. In order to detect changes in the surrounding environment information, this built-in sensor is attached to the embedded server of the out-home network. A processor for processing the information from the sensor is also attached. The processor is a small device equipped with a wireless transceiver to transfer such information, not as a means of communication but instead with the goal of gathering environment-related information. This is different from existing networks. The wireless sensor network consists of sensor nodes and gateway nodes. The former senses the surrounding environment and then process and transfer data about the events that they sense to sink nodes. The latter, the gateway nodes, gather information from the sensor nodes and transfer it to an external network, such as a TCP/IP network. Information moves both ways through these gateways, so the user can send a query to the sensor field through the sink nodes or receive the gathered data collected in the sensor field. The embedded server, equipped with a built-in sensor, acts as a sensor gateway server. The gateway protocol stack is shown in Fig. 2.[Fig. 1] The ubiquitous home network system model
   [Fig. 2] The gateway protocol stack in the embedded server
The embedded server sets and manages the values for the factors governing the network topology of the sensor network, which consists of sensor nodes, sink nodes, and a gateway. The factors governing the network topology include the maximum allowable number of child nodes per sensor node, the RF transmission power of the sensor nodes, the available wireless channels of the sensor nodes, and the wireless bandwidth available for the sensor nodes (e.g. the number of Guaranteed Time Slots of the IEEE 802.15.4 MAC). The sensor network suffers frequent changes in its network topology for a variety of reasons— the battery limitations of the sensor nodes, a disconnected wireless link, the addition of a new sensor node, the movement of sensor nodes, and so on. These changes are monitored in real time and they are displayed in an administrator-friendly graphic form. The sensor network is designed to manage the network topology by independently coping with the network environment with an automatic network restoration function. Either periodically or on an administrator’s command, the sensor nodes transfer to the server the information that is necessary for understanding the network topology. The information, which is assembled into a database, has the following content:
   ● An H/W-related table is defined to manage H/W information and the state of the sensor nodes in the server. ● Information about sensor network topology is saved and managed in the server. ● The sensor network supporting the mobility of sensor nodes and sink nodes manages the changes in the network topology with movement of the nodes. ● The sensor nodes save information about composition management and, upon request from the server, these nodes deliver this information to the server via the gateway. ● The sensor node and gateway save their IDs. The server synthetically saves and manages the IDs of the sensor nodes and the gateway. These IDs are provided in an administrator-friendly form (alias). For this, the network management system provides a mapping function between an ID and alias.
The server manages the location information of the sensor nodes. Such location information can be entered by the administrator or automatically reported to the server by the location-sensing function. The location information is saved and managed with a connection to the ID of the sensor node of the server. With this method, the embedded server, generally maintained by the home network management server, manages the entire external sensor network. The home network management server operates using the following modules:Information about the distances between node pairs is needed to determine the location of each sensor node. The RF signal strength at each sensor node is used to provide this distance information. ● A location-sensing message is delivered to the surrounding gateway nodes in the sink nodes. ● The cluster head delivers a signal to the individual
   sensor nodes associated with a gateway node by means of flooding. ● All the sensor nodes in a cluster broadcast messages to the surrounding nodes by means of flooding. Simultaneously, each of the nodes saves the outgoing node ID/incoming node ID and the received signal strength indicator (RSSI) with the message it received. ● As soon as the flooding is complete, the RSSI values of all of the nodes are saved. The nodes then send the saved RSSI information to the cluster head by means of a response message.
The time synchronization module has the function of synchronizing the time between the sink nodes and the sensor nodes that compose the sensor network. Message switching is provided to send time synchronization commands from sensor nodes at an upper level to those at a lower level, and time stamp message switching is provided between the sensor nodes at an upper level and those at a lower level. The differences between the clocks are calculated according to the time stamp information. The sensor nodes at the lower levels use the calculated clock differences to synchronize their clocks with those at the upper level.A list of neighbor sensor nodes is obtained by the sensor gateway. The list of sensor nodes specifies the type of the node (parent, child, and neighbor nodes), its relationship with other nodes within each network, and the address of each node and RSSI information. When obtaining the list information, the maximum desired number of sensor nodes is specified. Information about the sensor units mounted on each sensor node is imported. A function for importing the condition of the sensing node and sensing information from the network administrator is provided. Information about the sensor unit mounted on the sensor node is required to be set.
   When a problem arises in the network connection, the
network information is independently initialized and the surrounding network is connected by itself. A forced initialization can be performed with an administrator's command. The administrator may change the duty cycle of sensor nodes as necessary. This duty cycle determines the active state time and sleep state time of the sensor nodes. In the case of applications that periodically gather sensing data, the duty cycle of the sensor nodes is closely related to the sensing data gathering intervals. For a sensor network using the ZigBee protocol, the nodes performing the router function consume more power than the other nodes because of their packet switching function. If such power consumption continues, the nodes performing the router function will run out of battery power faster than the other nodes, reducing the life of the entire sensor network. To avoid this, it is necessary that the remaining battery capacity of a node be sensed. Then if a predefined critical value is reached, the routing function of that node is stopped, and other nodes are directed to take the place of the node with the weak battery. Our system was designed to use sensor nodes more efficiently by gathering large amounts of data by means of image processing, and to collect the surrounding environmental data by means of sensor nodes in order to make up for such a weak point in the sensing network. If the density of sensor nodes is too high—that is, if large numbers of sensor nodes are installed in the same area— this may lead to unnecessary sensing. On the other hand, if sensor nodes are installed accompanied by image sensing, the number of sensor nodes required can be greatly reduced.A model of the system as it was implemented is shown as Fig. 3. We considered having no internal home system but only an outdoor system as, in general, there are many cases of such outdoor systems. For this study we selected a parking lot control system, which is currently used quite often, and we implemented it as an outdoor system of home network systems. This ubiquitous network is very useful in providing a more flexible wireless home network service to utilize the surrounding information. As ubiquitous computing environments are being introduced, the systems require technology that is capable of supporting active services by sensing and utilizing situation information rather than simply supporting services in response to a user's demand. The RF modem chip used in the sensor nodes for this study can support the MAC of IEEE 802.15.4 and physical layers. However, for the network management pilot system, we designed and used a MAC and routing protocol that can support a sensor network in a tree form. In the tree-form sensor network we used a base-station-centric data-gathering routing protocol (BCDGP), with a TDMA mode used for the MAC protocol. The network protocols ported to each of the sensor nodes were the same. Various sensors were attached to the sensor nodes in order to monitor desired environmental factors or to control a certain unit. We developed a control unit (the hardware of the sensor node) and a program to handle various situations or events. Assume, for example, that sensor nodes to sense temperature, humidity, and illumination are distributed in a vinyl house used for growing vegetables or gardening. Then the nodes periodically check these three factors and report the sensed information to sink nodes at the preset times. The gateway that receives the information from the sink nodes regularly connects to a fixed server and transmits the data. If an emergency is sensed, steps that have been previously defined to cope with the emergency automatically take place. The sensor nodes should be designed taking into account these kinds of conditions. In this study, however, the motion sensor was selected from a variety of sensors with additional functions in the parking lot control system. Although we have not yet considered other functions in our design, if a outdoor system is required for other functions, suitable functions can be designed and made. For cars, we used a wireless model car with IrDA. This car was controlled primarily through image processing with a Web camera in an embedded server. We performed micro-adjustments using a sensor. The system is an automated parking system designed to take the burden of parking off the driver by automatically controlling the vehicle. To do this, images are taken through the webcam to provide information about the vehicle's location within the parking lot, and, based on that data, an appropriate parking location is found, and the vehicle is controlled using infrared rays. The main system activation methods are as follows. (1) At this point, take images of the vehicle within the parking lot to detect its location and direction. (2) Based on the information about the detected vehicle, prepare the movement signal and transmit it to the board. (3) Transmit the received signal to the vehicle.[Fig. 3] The implementation of the ubiquitous home network system modelConnection [Fig. 4] The hardware block diagramThe specification of the sensor node is shown in the table 1, the implemented sensor nodes are shown in Fig. 5. In the Fig. 6, in place of a random distribution of sensor nodes that are connected to their parent node, it is the initial flow of the operation. If the sensor node operates, the MCU attached to the nodes operates to boot from the program to read a specific area of memory. At this time, it performs the initialization or the set up their own data and the hardware. If all ready to operates, the sensor node will start the creation of the Ad Hoc network. If this can not create a network topology, it does not operate the function as the sensor nodes. Therefore, it repeats the same thing until this is done successfully. On the completion of the network setting, the sensing environment data are detected. Also to receive data from the child node in the structure of the network, it enters to the date mode in time. If it has to send their data, they can send data to the parent node to wait times for their turn on the RF in the allotted time and for their turn off the RF to reduce the power consumption.[Fig. 5] The implemented sensor nodes[Table 1] SENSOR NODE CHARACTERISTICS Unit Characteristics CPU Atmega 128 Mhz A/D 8 Bit Flash 256kb RAM 8Kbytes Radio Freq. 900Mhz Data Rate < 256Kbps Power 3.3V Radio Range. < 100m Power Source 2AA(1.5Vx2) Other I/O 8 bits Start end initialize Shut down Network configuration No (Connecting to parent node ) No Yes Is battery eliminated ? Did the network configuration succeed ? Wait for time-out No Yes Is The network _state Time out to sense ? Yes ok No ? Yes time out Turn RF on Yes to read from Get_Data_from _Sensors child Node ? Read_Data No time out Turn RF off to send to Yes Turn RF on parent Node ? No Send_Data Turn RF off No Is battery eliminated? Yes Shut down end [Fig. 6] The operation flowchart of the sensor nodeThe image-processing system is and composed of two parts, a real-time image-tracking system and a sensing system that uses motion sensors. When images are acquired from a CCD camera, the analog signals are used to create image signals, and these are saved in the buffer by making into a cooperating Frame Grabber within embedded server equipped with real-time image-tracking program. Afterwards, the image of the target in the camera is detected with a moving-target detector, the target to be tracked is selected, and the reference and current images are transmitted to the initial model creator. The initial model creator obtains regional information using the current image and creates the initial model. From this initial model, a coordinate model that obtains an approximate center location is created. Sensing data from the location information, the model information, and the sensor node of the target are combined to provide a precise location, and this information is used to control outside devices. These processes are depicted as Fig, 7. At this point, the contents of system activation are as follows: - Images of the parking lot and the vehicle inside it are taken with a webcam connected to a PC. - With the obtained images, the location and direction of the vehicle inside the parking lot are detectedPC Moving Frame CCD Target Grabber Camera Detector Initial Coordinate Model Sensing Model Model Update Data Model Controlling Information The car [Fig. 7] The image processing block diagram
   using OpenCV library. - Based on the detected information, the direction that the vehicle is moving is determined and transmitted to the board using socket communication.
The hardware block diagram for the actual system as used is shown in Figure 5. The vehicle control is done according to following procedure: - A vehicle motion signal is received from the PC via socket communication. - The received data is transmitted to the IrDA. - PC/104 interface - Amplification circuit using transistor - IrDA running application being run in the boardIn Fig. 8, it represents the configuration of the module to represent the sensor data received from a gateway to the Web. This system operates three applications running at the same time, provides a single service finally. Each operating application is as follows.
   (1) Sensing Application (2) DB Server (3) Web Server
Sensing applications receive data from a sensor gateway connected to the Internet, save the received data from the DB Server to the DB. The DB server stores the sensing information by changing the fixed structure. The web server operates the GUI output of the web type to can recognize easily user as the web browser connect to the server via a web server with receiving the data from the DB server.
   Sensor Gateway Sensor Gateway Application R/W R/W R/W R/W Standard Database Modem Voice Files Library In/Out R/W Server Library R/W R/W Device Kernel(Linux) R/W Drivers R/W R/W R/W Hardware (ARM Board) Serial Port LAN R/W R/W Sink Module (Sensor Node) [Fig. 8] The operation of the ubiquitous home network system as implemented
Fig. 9 and Fig. 10 show the operation of the ubiquitous home sensor network as implemented.
   [Fig. 9] The operation of the ubiquitous home network system as implemented

   [Fig. 10] The operation of the embedded server with the gateway sensor node
Primitive messages for information exchange between the server and the mobile terminal are defined. Each message is divided into a Get command set that obtains information from the server and a Set command set that sets a certain value in the server. The primitive messages are: Get Sensor Network Lists (Request, Response, Indication, Confirm): Message to get a list of sensor networks accessible from the server Get Node Lists (Request, Response, Indication, Confirm): Message to get a list of the sensor nodes present in a certain sensor network Get Node Info (Request, Response, Indication, Confirm): Message to get information about a certain sensor node Get Node Data (Request, Response, Indication, Confirm): Message to receive data about a certain sensor node Get G/W Info (Request, Response, Indication, Confirm): Message to get information about the sensor gateway that manages a certain sensor network Set Data for a Sensor Node (Request, Response, Indication, Confirm): Message to set and control information about a certain sensor node Set G/W Info (Request, Response, Indication, Confirm): Message to set and control information about a certain sensor gateway Status Message: Message to check the progress of the requested job
   (a)

   (b)

   (c) [Fig. 11] The message flows between the home network management server and the embedded server with the gateway sensor nodel
Communications between a home network management server and an embedded server are carried out with these primitive messages. Each message is properly used according to the job to be performed. The procedure that is followed in performing the job is shown in Figure 11. This is the protocol of the operation of the implemented system. This is the operation to control a car through IrDA in the embedded server with a Web camera. Then, most of the control of the car is exerted through image processing. A very small part, if any, is adjusted via the sensor nodes in the parking lot. For sensor nodes arranged for implementing the system, the entire sensor network is set with the sensor gateway node as the coordinator. Logical 16-bit address information is assigned to each sensor node. Therefore, the action of the embedded system carrying sensor gateway nodes shall be done preferentially. The sensor network topology which was formed automatically could be identified through the LED actuating signal of each arranged sensor node by turning on the power of all the sensor nodes arranged after all ready actions of the sensor gateway are finished. Once the network is formed centered on sensor gateway nodes, the data are transferred to the sensor gateway nodes by turning on the RF mounted on each node every 2 seconds along the route of the network topology, which is essentially in a tree form.In order to evaluate the performance of our algorithm, we simulated a routing scheme of the implemented system. We used NS2 and simulation programs written in the Visual-C++ programming language to evaluate the performance of our scheme. We randomly generated networks with a diameter of 100m X 100m, each having approximately 5 to 300 nodes. We repeated the simulations for the same network, with one base station in the center, 10 times. In the case of our algorithm, the round time increased as the number of wireless embedded nodes (WENs) increased from 1 to 10. Our aim was to reduce the round times and shorten the hop counts in order to provide for a more efficient use of energy. We simulated the BCDGP with wireless sensor nodes of 0dB and WEN of 4 dB. And the radio ranges of wireless sensor are 10m, WENs’ radio ranges are 40m.
   700 With 1 WEN 600 With 2 WEN With 3 WEN With 4 WEN 500 With 5 WEN With 6 WEN With 7 WEN Times(ms) 400 With With 8 9 WEN WEN 300 With 10 WEN Round 200 100 0 0 20 40 60 80 100 120 The number of sensor nodes [Fig. 12] Round Times vs. the changes of sensor nodes
The aim is efficient transmission of all the data to the base station so that the lifetime of the network is maximized in terms of rounds, where a round is defined as the process of gathering all the data from sensor nodes to the base station, regardless of how much time it takes as shown Fig. 12. Fig. 13 shows the hop counts depending on the number of WENs and sensor nodes. As shown in Fig. 13, an increase in the number of WENs reduces the hop counts. Reducing the hop counts leads to a reduction of the remaining energy of the wireless sensor nodes. Therefore, BCDGP is the more efficient energy consumption algorithm.7 62 3 4 5 6 7count 5 Hop 4es3
   [Fig. 13] Round Times vs. the changes of sensor nodes in the best case
Home networks provides users with a variety of information services. By adding various types of sensor data to the home network, it is possible to provide more and better services. However, existing home networks limit the available range of services by concentrating primarily on multimedia options. There has been excellent progress toward controlling home appliances through communication across home networks, but there has so far been little research and development aimed at controlling outdoor appliances. The functions of outdoor appliances are generally different in various ways from home appliances, and sensors with one or two functions, rather than many functions, can be used to compose an outdoor system. Because they have a limited number of functions, these sensors are generally less expensive than the sensors used indoors. In this study, we choose to focus on motion sensors, as this type of sensor has many current uses. We chose to use the motion sensors in a parking lot control system, which was applied and implemented in a home network. To carry this out, we defined a protocol to manage the network as part of a ubiquitous sensor network, and we implemented it for the wireless home network in this study. Although a network management system in a USN related to this study has researched for the purpose of other independent projects, the protocol interface and message system have not clearly been defined yet such as making the interface. Therefore USN network management development needs to be developed further in terms of the management of faults, composition, power, and applications. Based on the required management items, we defined a PI (Protocol Interface) for efficient and systematic communication between networks, and the subsequent message was described. To verify the performance of the protocol interface we designed, we designed and implemented the necessary units (sensor nodes, sensor gateway, and server) and used them to test the proposed system. This system photographs contexts within a certain space rather than the vehicle itself, it detects changes within the received images that are used as an intruder-monitoring system, it photographs road conditions in front of the vehicle, and it controls the vehicle accordingly. Thus the system is capable of being expanded into an automated navigational system or telematics network. It also could be expanded into an automated navigational system that controls vehicles using GPS and navigation modules besides the image-analyzing method. However, research on ubiquitous network systems today is around systems with a single function and not around different devices or multiple functions. Thus, we focused our attention on a more efficient and highly applicable system based on a single function.