(en)In accordance with an example embodiment a method, apparatus and computer program product are provided. The method comprises receiving, at an apparatus, a sampled data associated with an activity from one or more sensors wirelessly coupled to the apparatus. The sampled data is generated at the one or more sensors based on a compressive sampling of an activity data associated with the activity. The compressive sampling is performed based on a sampling information. The activity is classified based at least on the sampled data. An error associated with the classification of the activity is determined. The sampling information is updated or retained based on a comparison of the error with a threshold error. The updated sampling information is utilized for generating an updated sampled data. The updated sampled data facilitates in reclassification of the activity.

1.ApplicationNumber: US-201414333471-A
1.PublishNumber: US-2015039260-A1
2.Date Publish: 20150205
3.Inventor: NISKANEN ANTTI
WABNIG JOACHIM

4.Inventor Harmonized: NISKANEN ANTTI(GB)
WABNIG JOACHIM(GB)

5.Country: US
6.Claims:
(en)In accordance with an example embodiment a method, apparatus and computer program product are provided. The method comprises receiving, at an apparatus, a sampled data associated with an activity from one or more sensors wirelessly coupled to the apparatus. The sampled data is generated at the one or more sensors based on a compressive sampling of an activity data associated with the activity. The compressive sampling is performed based on a sampling information. The activity is classified based at least on the sampled data. An error associated with the classification of the activity is determined. The sampling information is updated or retained based on a comparison of the error with a threshold error. The updated sampling information is utilized for generating an updated sampled data. The updated sampled data facilitates in reclassification of the activity.

7.Description:
(en)TECHNICAL FIELD
Various implementations relate generally to method, apparatus, and computer program product for activity recognition.
BACKGROUND
In recent years, there has been a tremendous growth in the technology related to context-awareness based applications such as activity recognition. Activity recognition helps in identifying user needs based on sensing of human activities. Applications of activity recognition may include various real-life human centric problems/situations such as healthcare, elder-care, household activities, and the like. It may be desirable to provide reliable, cost-effective and energy efficient solutions for such human centric problems through activity recognition.
SUMMARY OF SOME EMBODIMENTS
Various aspects of example embodiments are set out in the claims.
In a first aspect, there is provided a method comprising: receiving, at an apparatus, a sampled data associated with an activity from one or more sensors wirelessly coupled to the apparatus, the sampled data being generated at the one or more sensors based on a compressive sampling of an activity data associated with the activity, the compressive sampling being performed based on a sampling information; classifying the activity based at least on the sampled data; determining an error associated with the classification of the activity; and updating or retaining the sampling information based on a comparison of the error with a threshold error.
In a second aspect, there is provided a method comprising: receiving, at a sensor, a sampling information from an apparatus wirelessly coupled to the sensor; performing compressive sampling of an activity data associated with an activity based at least on the sampling information and a sampling criteria for generating a sampled data associated with the activity; and sending the sampled data to the apparatus for facilitating classification of the activity.
In a third aspect, there is provided an apparatus comprising at least one processor; and at least one memory comprising computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least perform: receive, at an apparatus, a sampled data associated with an activity from one or more sensors wirelessly coupled to the apparatus, the sampled data being generated at the one or more sensors based on a compressive sampling of an activity data associated with the activity, the compressive sampling being performed based on a sampling information; classify the activity based at least on the sampled data; determine an error associated with the classification of the activity; and update or retain the sampling information based on a comparison of the error with a threshold error.
In a fourth aspect, there is provided an apparatus comprising at least one processor; and at least one memory comprising computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least perform: receive, at the apparatus, a sampling information from another apparatus wirelessly coupled to the apparatus; perform compressive sampling of an activity data associated with an activity based at least on the sampling information and a sampling criteria for generating a sampled data associated with the activity; and send the sampled data to the another apparatus for facilitating classification of the activity.
In a fifth aspect, there is provided a computer program product comprising at least one computer-readable storage medium, the computer-readable storage medium comprising a set of instructions, which, when executed by one or more processors, cause an apparatus to at least perform: receive, at an apparatus, a sampled data associated with an activity from one or more sensors wirelessly coupled to the apparatus, the sampled data being generated at the one or more sensors based on a compressive sampling of an activity data associated with the activity, the compressive sampling being performed based on a sampling information; classify the activity based at least on the sampled data; determine an error associated with the classification of the activity; and update or retain the sampling information based on a comparison of the error with a threshold error.
In a sixth aspect, there is provided a computer program product comprising at least one computer-readable storage medium, the computer-readable storage medium comprising a set of instructions, which, when executed by one or more processors, cause an apparatus to at least perform: receive, at the apparatus, a sampling information from another apparatus wirelessly coupled to the apparatus; perform compressive sampling of an activity data associated with an activity based at least on the sampling information and a sampling criteria for generating a sampled data associated with the activity; and send the sampled data to the another apparatus for facilitating classification of the activity.
In a seventh aspect, there is provided an apparatus comprising: means for receiving, at an apparatus, a sampled data associated with an activity from one or more sensors wirelessly coupled to the apparatus, the sampled data being generated at the one or more sensors based on a compressive sampling of an activity data associated with the activity, the compressive sampling being performed based on a sampling information; means for classifying the activity based at least on the sampled data; means for determining an error associated with the classification of the activity; and means for updating or retaining the sampling information based on a comparison of the error with a threshold error.
In an eighth aspect, there is provided an apparatus comprising: means for receiving, at the apparatus, a sampling information from another apparatus wirelessly coupled to the apparatus; means for performing compressive sampling of an activity data associated with an activity based at least on the sampling information and a sampling criteria for generating a sampled data associated with the activity; and means for sending the sampled data to the another apparatus for facilitating classification of the activity.



BRIEF DESCRIPTION OF THE FIGURES
Various embodiments are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which:
FIG. 1 illustrates an exemplary system for activity recognition in accordance with an example embodiment;
FIG. 2 illustrates a device in accordance with an example embodiment;
FIG. 3 illustrates an apparatus for activity recognition in accordance with an example embodiment;
FIG. 4 illustrates another apparatus for activity recognition in accordance with another example embodiment;
FIG. 5 illustrates an example of compressive sampling of an activity data associated with activity recognition, in accordance with an example embodiment;
FIG. 6 illustrates a sequence of transmission of sampled data in accordance with an embodiment;
FIG. 7 is a flowchart depicting an example method for activity recognition, in accordance with an example embodiment;
FIG. 8 is a flowchart depicting an example method for activity recognition, in accordance with another example embodiment; and
FIG. 9 is a flowchart depicting an example method for activity recognition, in accordance with yet another example embodiment.



DETAILED DESCRIPTION
Example embodiments and their potential effects are understood by referring to FIGS. 1 through 9 of the drawings.
FIG. 1 illustrates an exemplary system 100 for performing activity recognition in accordance with an example embodiment. In an example embodiment, the system 100 may be configured to facilitate activity recognition based on sensing of an activity data associated with an activity, for example, a human activity. In particular, the purpose of activity recognition is to infer human behaviors from the activity data that may be acquired through one or more sensors in a given setting, based on which various decisions may be made. In an example embodiment, the information about the user's activities may be utilized to provide activity-dependent user experiences. For instance, based on activity recognition, if the user is determined to be running, then a user interface (UI) of a device coupled with the activity recognition sensors may adjust to a more accessible format.
As illustrated in FIG. 1 , the system 100 includes one or more sensors, for example, sensors 102 , 104 , 106 , communicably coupled with a device for example, a device 108 . In an embodiment, the device 108 may be any kind of equipment that is able to communicate with the one or more sensors. In an embodiment, the device may be a smartphone, a laptop, a mobile phone a personal digital assistant (PDA), and the like. In an embodiment the one or more sensors may be coupled with the device through a communication path. In an embodiment, the communication path may be a wireless link, for example, a wireless link 110 . In an embodiment, the wireless link 110 may include a radio link access network of a wireless communication network. Examples of wireless communication network may include, but are not limited to a cellular communication network. The communication path may additionally include other elements of a wireless communication network and even elements of a wire-line communication network to which the wireless communication network is coupled.
The one or more sensors may be configured to acquire activity data associated with a human activity. In an example embodiment, the activity data may include measurement data associated with an activity, for example, motion, temperature, location, and the like. In an embodiment, the one or more sensors may be configured to sample the activity data and in response generate a sampled data. In an embodiment, the sampled data may be generated by performing compressive sampling on the activity data using random basis vectors.
As used herein, compressive sampling/sensing facilitates in data compression and reconstruction. According to compressive sensing theorem, when an original signal can be expressed as a large number of zero or nearly zero values and a small number of non-zero values, the signals is said to be “sparse”. Compressive sensing exploits the fact that many natural signals are sparse or compressible and as such these signals have concise representations when expressed in the appropriate basis. Sampling at a rate below the Nyquist frequency in a basis that is “incoherent” to the “sparse” basis facilitates in collecting all the relevant data and allows reconstruction of the original signal. As such, compressing sensing facilities in reducing an amount of activity data being sampled by the one or more sensors, thereby reducing computational requirement at the one or more sensors. The sampled data that is being generated by the one or more sensors may hereinafter be referred to sampled data. As such, the terms ‘sampled data’ and ‘compressively sampled data’ may be used interchangeably throughout the description.
In an embodiment, the sampled data generated by the one or more sensors may be sent to the device 108 . In an embodiment, the sampled data may be sent to the device 108 via a wireless link, such as the link 110 . Based on the sampled data received from one or more sensors, the device 108 classifies an “activity” of the user, and assigns a label to the activity. In an embodiment, the term activity may include various examples such as running, walking, climbing up or down the stairs, jogging, cooking, opening a refrigerator, and the like. In general, any condition depicting an action and/or a movement by the user may be referred to as an activity, and accordingly the cited examples should not be construed as limiting to the present disclosure.
In an embodiment, the sampled data may be directly utilized for performing classification, without performing decompression of the sampled data. The preclusion of a need to decompress the sampled data prior to classification is advantageous in lowering the computational processing and power requirement of the device 108 . Additionally, since the data is compressively sampled at the one or more sensors, it can be economically transmitted and stored on the device.
In an embodiment, the information associated with the activity may be used to provide activity-dependent user experiences. For instance, if the user is considered to be running, a user interface of the device 108 may adjust to a format that may be easily accessed by the user. In an example embodiment, only a selected one or more sensors may sample respective activity data associated with the activity of the user, and send the respective sampled data to the device 108 . In an alternate example embodiment, multiple sensors coupled with the device 108 may sample the respective activity data and send the respective sampled data to the device 108 . It will be understood that herein the one or more sensors (or multiple sensors) refers to various distinct sensors that are configured to sense different physical quantities. In an embodiment, the one or more sensors that are configured to send the corresponding sampled data to the device 108 may operate in synchronization. For example, when multiple sensors are operating for sensing and collecting the data, the timing of the multiple sensors may be synchronized.
FIG. 2 illustrates a device 200 in accordance with an example embodiment. In an embodiment, the device 200 may be an example of the device 108 (refer to FIG. 1 ). It should be understood, however, that the device 200 as illustrated and hereinafter described is merely illustrative of one type of device that may benefit from various embodiments, therefore, should not be taken to limit the scope of the embodiments. As such, it should be appreciated that at least some of the components described below in connection with the device 200 may be optional and thus in an example embodiment may include more, less or different components than those described in connection with the example embodiment of FIG. 2 . The device 200 could be any of a number of types of mobile electronic devices, for example, portable digital assistants (PDAs), pagers, mobile televisions, gaming devices, cellular phones, all types of computers (for example, laptops, mobile computers or desktops), cameras, audio/video players, radios, global positioning system (GPS) devices, media players, mobile digital assistants, or any combination of the aforementioned, and other types of communications devices.
The device 200 may include an antenna 202 (or multiple antennas) in operable communication with a transmitter 204 and a receiver 206 . The device 200 may further include an apparatus, such as a controller 208 or other processing device that provides signals to and receives signals from the transmitter 204 and receiver 206 , respectively. The signals may include signaling information in accordance with the air interface standard of the applicable cellular system, and/or may also include data corresponding to user speech, received data and/or user generated data. In this regard, the device 200 may be capable of operating with one or more air interface standards, communication protocols, modulation types, and access types. By way of illustration, the device 200 may be capable of operating in accordance with any of a number of first, second, third and/or fourth-generation communication protocols or the like. For example, the device 200 may be capable of operating in accordance with second-generation (2G) wireless communication protocols IS-136 (time division multiple access (TDMA)), GSM (global system for mobile communication), and IS-95 (code division multiple access (CDMA)), or with third-generation (3G) wireless communication protocols, such as Universal Mobile Telecommunications System (UMTS), CDMA1000, wideband CDMA (WCDMA) and time division-synchronous CDMA (TD-SCDMA), with 3.9G wireless communication protocol such as evolved-universal terrestrial radio access network (E-UTRAN), with fourth-generation (4G) wireless communication protocols, or the like. As an alternative (or additionally), the device 200 may be capable of operating in accordance with non-cellular communication mechanisms. For example, computer networks such as the Internet, local area network, wide area networks, and the like; short range wireless communication networks such as Bluetooth® networks, Zigbee® networks, Institute of Electric and Electronic Engineers (IEEE) 802.11x networks, and the like; wireline telecommunication networks such as public switched telephone network (PSTN).
The controller 208 may include circuitry implementing, among others, audio and logic functions of the device 200 . For example, the controller 208 may include, but are not limited to, one or more digital signal processor devices, one or more microprocessor devices, one or more processor(s) with accompanying digital signal processor(s), one or more processor(s) without accompanying digital signal processor(s), one or more special-purpose computer chips, one or more field-programmable gate arrays (FPGAs), one or more controllers, one or more application-specific integrated circuits (ASICs), one or more computer(s), various analog to digital converters, digital to analog converters, and/or other support circuits. Control and signal processing functions of the device 200 are allocated between these devices according to their respective capabilities. The controller 208 thus may also include the functionality to convolutionally encode and interleave message and data prior to modulation and transmission. The controller 208 may additionally include an internal voice coder, and may include an internal data modem. Further, the controller 208 may include functionality to operate one or more software programs, which may be stored in a memory. For example, the controller 208 may be capable of operating a connectivity program, such as a conventional Web browser. The connectivity program may then allow the device 200 to transmit and receive Web content, such as location-based content and/or other web page content, according to a Wireless Application Protocol (WAP), Hypertext Transfer Protocol (HTTP) and/or the like. In an example embodiment, the controller 108 may be embodied as a multi-core processor such as a dual or quad core processor. However, any number of processors may be included in the controller 108 .
The device 200 may also comprise a user interface including an output device such as a ringer 210 , an earphone or speaker 212 , a microphone 214 , a display 216 , and a user input interface, which may be coupled to the controller 208 . The user input interface, which allows the device 200 to receive data, may include any of a number of devices allowing the device 200 to receive data, such as a keypad 218 , a touch display, a microphone or other input device. In embodiments including the keypad 218 , the keypad 218 may include numeric (0-9) and related keys (#, *), and other hard and soft keys used for operating the device 200 . Alternatively or additionally, the keypad 218 may include a conventional QWERTY keypad arrangement. The keypad 218 may also include various soft keys with associated functions. In addition, or alternatively, the device 200 may include an interface device such as a joystick or other user input interface. The device 200 further includes a battery 220 , such as a vibrating battery pack, for powering various circuits that are used to operate the device 200 , as well as optionally providing mechanical vibration as a detectable output.
In an example embodiment, the device 200 includes a media capturing element, such as a camera, video and/or audio module, in communication with the controller 108 . The media capturing element may be any means for capturing an image, video and/or audio for storage, display or transmission. In an example embodiment, the media capturing element is a camera module 222 which may include a digital camera capable of forming a digital image file from a captured image. As such, the camera module 222 includes all hardware, such as a lens or other optical component(s), and software for creating a digital image file from a captured image. Alternatively, or additionally, the camera module 222 may include the hardware needed to view an image, while a memory device of the device 100 stores instructions for execution by the controller 208 in the form of software to create a digital image file from a captured image. In an example embodiment, the camera module 222 may further include a processing element such as a co-processor, which assists the controller 208 in processing image data and an encoder and/or decoder for compressing and/or decompressing image data. The encoder and/or decoder may encode and/or decode according to a JPEG standard format or another like format. For video, the encoder and/or decoder may employ any of a plurality of standard formats such as, for example, standards associated with H.261, H.262/MPEG-2, H.263, H.264, H.264/MPEG-4, MPEG-4, and the like. In some cases, the camera module 222 may provide live image data to the display 216 . In an example embodiment, the display 216 may be located on one side of the device 200 and the camera module 222 may include a lens positioned on the opposite side of the device 200 with respect to the display 216 to enable the camera module 222 to capture images on one side of the device 200 and present a view of such images to the user positioned on the other side of the device 200 .
The device 200 may further include a user identity module (UIM) 224 . The UIM 224 may be a memory device having a processor built in. The UIM 224 may include, for example, a subscriber identity module (SIM), a universal integrated circuit card (UICC), a universal subscriber identity module (USIM), a removable user identity module (R-UIM), or any other smart card. The UIM 224 typically stores information elements related to a mobile subscriber. In addition to the UIM 224 , the device 200 may be equipped with memory. For example, the device 200 may include volatile memory 226 , such as volatile random access memory (RAM) including a cache area for the temporary storage of data. The device 200 may also include other non-volatile memory 228 , which may be embedded and/or may be removable. The non-volatile memory 228 may additionally or alternatively comprise an electrically erasable programmable read only memory (EEPROM), flash memory, hard drive, or the like. The memories may store any number of pieces of information, and data, used by the device 200 to implement the functions of the device 200 .
FIG. 3 illustrates an apparatus 300 for activity recognition, in accordance with an example embodiment. The apparatus 300 for activity recognition may be employed, for example, in the device 200 of FIG. 2 . However, it should be noted that the apparatus 300 , may also be employed on a variety of other devices both mobile and fixed, and therefore, embodiments should not be limited to application on devices such as the device 200 of FIG. 2 . Alternatively, embodiments may be employed on a combination of devices including, for example, those listed above. Various embodiments may be embodied wholly at a single device, (for example, the device 200 ). It should also be noted that some of the devices or elements described below may not be mandatory and thus some may be omitted in certain embodiments.
In an embodiment, the apparatus 300 includes or otherwise is in communication with at least one processor 302 and at least one memory 304 . Examples of the at least one memory 304 include, but are not limited to, volatile and/or non-volatile memories. Some examples of the volatile memory include, but are not limited to, random access memory, dynamic random access memory, static random access memory, and the like. Some example of the non-volatile memory includes, but are not limited to, hard disks, magnetic tapes, optical disks, programmable read only memory, erasable programmable read only memory, electrically erasable programmable read only memory, flash memory, and the like. The memory 304 may be configured to store information, data, applications, instructions or the like for enabling the apparatus 300 to carry out various functions in accordance with various example embodiments. For example, the memory 304 may be configured to buffer a sampling information comprising a basis information and a selection information for processing by the processor 302 . Additionally or alternatively, the memory 304 may be configured to store instructions for execution by the processor 302 .
An example of the processor 302 may include the controller 308 . The processor 302 may be embodied in a number of different ways. The processor 302 may be embodied as a multi-core processor, a single core processor; or combination of multi-core processors and single core processors. For example, the processor 302 may be embodied as one or more of various processing means such as a coprocessor, a microprocessor, a controller, a digital signal processor (DSP), processing circuitry with or without an accompanying DSP, or various other processing devices including integrated circuits such as, for example, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a microcontroller unit (MCU), a hardware accelerator, a special-purpose computer chip, or the like. In an example embodiment, the multi-core processor may be configured to execute instructions stored in the memory 304 or otherwise accessible to the processor 302 . Alternatively or additionally, the processor 302 may be configured to execute hard coded functionality. As such, whether configured by hardware or software methods, or by a combination thereof, the processor 302 may represent an entity, for example, physically embodied in circuitry, capable of performing operations according to various embodiments while configured accordingly. For example, if the processor 302 is embodied as two or more of an ASIC, FPGA or the like, the processor 302 may be specifically configured hardware for conducting the operations described herein. Alternatively, as another example, if the processor 302 is embodied as an executor of software instructions, the instructions may specifically configure the processor 302 to perform the algorithms and/or operations described herein when the instructions are executed. However, in some cases, the processor 302 may be a processor of a specific device, for example, a mobile terminal or network device adapted for employing embodiments by further configuration of the processor 302 by instructions for performing the algorithms and/or operations described herein. The processor 302 may include, among other things, a clock, an arithmetic logic unit (ALU) and logic gates configured to support operation of the processor 302 .
A user interface 306 may be in communication with the processor 302 . Examples of the user interface 306 include, but are not limited to, input interface and/or output user interface. The input interface is configured to receive an indication of a user input. The output user interface provides an audible, visual, mechanical or other output and/or feedback to the user. Examples of the input interface may include, but are not limited to, a keyboard, a mouse, a joystick, a keypad, a touch screen, soft keys, and the like. Examples of the output interface may include, but are not limited to, a display such as light emitting diode display, thin-film transistor (TFT) display, liquid crystal displays, active-matrix organic light-emitting diode (AMOLED) display, a microphone, a speaker, ringers, vibrators, and the like. In an example embodiment, the user interface 306 may include, among other devices or elements, any or all of a speaker, a microphone, a display, and a keyboard, touch screen, or the like. In this regard, for example, the processor 302 may comprise user interface circuitry configured to control at least some functions of one or more elements of the user interface 306 , such as, for example, a speaker, ringer, microphone, display, and/or the like. The processor 302 and/or user interface circuitry comprising the processor 302 may be configured to control one or more functions of one or more elements of the user interface 306 through computer program instructions, for example, software and/or firmware, stored on a memory, for example, the at least one memory 304 , and/or the like, accessible to the processor 302 .
In an example embodiment, the apparatus 300 may include an electronic device. Some examples of the electronic device include communication device, media capturing device, media capturing device with communication capabilities, computing devices, and the like. Some examples of the communication device may include a mobile phone, a personal digital assistant (PDA), and the like. Some examples of computing device may include a laptop, a personal computer, and the like. In an example embodiment, the electronic device may include a user interface, for example, the UI 206 , having user interface circuitry and user interface software configured to facilitate a user to control at least one function of the electronic device through use of a display and further configured to respond to user inputs. In an example embodiment, the electronic device may include a display circuitry configured to display at least a portion of the user interface of the electronic device. The display and display circuitry may be configured to facilitate the user to control at least one function of the electronic device.
In an example embodiment, the electronic device may be embodied as to include a transceiver. The transceiver may be any device operating or circuitry operating in accordance with software or otherwise embodied in hardware or a combination of hardware and software. For example, the processor 302 operating under software control, or the processor 302 embodied as an ASIC or FPGA specifically configured to perform the operations described herein, or a combination thereof, thereby configures the apparatus or circuitry to perform the functions of the transceiver. The transceiver may be configured to receive a sampled data from one or more sensors.
These components ( 302 - 306 ) may communicate with each other via a centralized circuit system 308 . The centralized circuit system 308 may be configured to, among other things, provide or enable communication between the components ( 302 - 306 ) of the apparatus 300 . In certain embodiments, the centralized circuit system 308 may be a central printed circuit board (PCB) such as a motherboard, main board, system board, or logic board. The centralized circuit system 308 may also, or alternatively, include other printed circuit assemblies (PCAs) or communication channel media.
In an example embodiment, the processor 302 is configured to, with the content of the memory 304 , and optionally with other components described herein, to cause the apparatus 300 to perform activity recognition for an activity. In an example embodiment, the activity may be a human activity. For example, the activity may be walking, jogging, running, climbing-up, climbing down, and the like. Various other real-world activities may be recognized by the apparatus 300 without limiting the scope of the disclosure.
In an example embodiment, the processor 302 is configured to, with the content of the memory 304 , and optionally with other components described herein, to cause the apparatus 300 to generate a basis matrix associated with a space. In an embodiment, the basis matrix may be a complete basis matrix. The complete basis matrix may be configured to span an entire space, such that, any vector on the space can be written as a linear combination of the n vectors of the basis. In an embodiment, the size of the complete basis matrix may be adjustable. In an embodiment, the basis matrix may be generated by a random number generator providing random numbers evenly distributed in the interval [−1, 1]. In order to make a set of n vectors into a complete basis, the randomly generated vectors may be orthonormalised. In an example embodiment, the orthonormal random matrix may be generated based on a QR decomposition method, wherein a matrix is decomposed into a product of an orthogonal matrix (Q) and an upper triangular matrix (R). In an example embodiment, the randomly generated vectors may be orthonormalised based on a Gram-Schmidt algorithm. This basis corresponds to an orthonormal square matrix. In an embodiment, a processing means may be configured to generate the matrix of complete basis. An example of the processing means may include the processor 302 , which may be an example of the controller 208 .
In an example embodiment, the processor 302 is configured to, with the content of the memory 304 , and optionally with other components described herein, to cause the apparatus 300 to send a sampling information to the one or more sensors for facilitating the one or more sensors to generate the sampled data. In an embodiment, the sampling information includes the basis matrix and a selection information for selecting a subset of the basis matrix. The subset of the basis matrix may include a set of basis vectors that may be utilized by the one or more sensors for generating the sampled data. In an embodiment, a processing means may be configured to send the sampling information to the one or more sensors. An example of the processing means may include the processor 302 , which may be an example of the controller 208 . In an embodiment, the sampling information may be sent to the one or more sensors over a wireless link.
In an embodiment, the one or more sensors may be configured to receive the basis matrix and process the activity data based on the basis matrix and the selection information. Based on the processing of the activity data, the one or more sensors may be configured to generate a sampled data. In an embodiment, the sampled data may be generated based on compressive sampling of the activity data. The description and functionality of the one or more sensors for generating the sampled data is explained further with reference to FIGS. 4 and 5 .
In an example embodiment, the processor 302 is configured to, with the content of the memory 304 , and optionally with other components described herein, to cause the apparatus 300 to receive the sampled data associated with the activity from the one or more sensors. In an embodiment, the apparatus 300 is configured to receive the sampled data until a frame of the sampled data is received. In an embodiment, the size of the frame may be about 0.1s-10s. In an embodiment, if multiple sensors are sending a respective sampled data to the apparatus 300 , then the sampled data received from multiple sensors may be arranged in a sequence of their respective reception at the apparatus 300 . The arrangement of the sampled data received from the one or more sensors is explained in further detail with reference to FIG. 6 .
In an embodiment, the apparatus 300 is configured to classify the activity based on the sampled data sent by the one or more sensors. In an embodiment, the processor 302 is configured to, with the content of the memory 304 , and optionally with other components described herein, to cause the apparatus 300 to apply a classification rule to the sampled data. In an embodiment, the classification rule/model for supervised learning may be trained by utilizing a labeled activity data. In an embodiment, the labeled activity data may be produced based on the sampling information that is utilized in the performing random projection. In an embodiment, the training set with labels may be represented in the basis that is utilized for sampling of the activity data at the one or more sensors. In an embodiment, the classification rule may utilize one of a quadratic discriminant mechanism, support vector machine, decision tree, neural net and a Boltzmann machine mechanisms for labeling the activity data.
In another embodiment, the labels in the training set may be provided initially (e.g. as in “factory settings”) in a basis so that it may be convenient to compute the training set in any undercomplete basis, for example, the labels utilized for compressive sampling of the activity data. In another embodiment, the labels associated with the training set may be deduced from other data. For instance, the labels may be available for partial activity data when the activity data is sampled in a complete basis. The labels may be used to generate a training set of undercomplete (projected) data with additional sensors included. In yet another embodiment, the labels may be obtained based on an unsupervised learning. For example, various identify clusters (e.g. k-means algorithm) may be generated and labels may be obtained by “asking” information from the user. For instance, in response to detection of a motion activity, a query such as ‘were you running?’ may be asked.
In an embodiment, a processing means may be configured to classify the activity based on the sampled data and the classification rule. An example of the processing means may include the processor 302 , which may be an example of the controller 208 . The classification of the activity data may provide an activity class (e.g. running, sitting, walking, etc.) as output. It will be understood herein that instead of decompressing the sampled data, the sampled data may be directly utilized for the purpose of classification. An advantage of utilizing the sampled data directly (without decompression) is that the computational power associated with decompression of data is reduced.
In an example embodiment, the processor 302 is configured to, with the content of the memory 304 , and optionally with other components described herein, to cause the apparatus 300 to store a first label into which the activity is classified. In an embodiment, a memory means may be configured to store the first label. An example of the processing means may include the memory means 304 , which may be an example of the non-volatile memory 228 .
In an embodiment, the output of the classification may contain error due to the presence of noise in the measurement of activity data when the activity data is acquired by the one or more sensors. In an embodiment, the error in the sampled data may lead to classification of the activity into an incorrect label. In an embodiment, the processor 302 is configured to, with the content of the memory 304 , and optionally with other components described herein, to cause the apparatus 300 to determine the error associated with the classification of the activity data. In an embodiment, since the activities performed by the users do not undergo substantial fluctuation over a very short time scale (for example, seconds), a variance in the output of the classification output may be utilized for detection of any errors in the classification. In an embodiment, a processing means may be configured to determine the error associated with the sampled data. An example of the processing means may include the processor 302 , which may be an example of the controller 208 .
In an embodiment, the processor 302 is configured to, with the content of the memory 304 , and optionally with other components described herein, to cause the apparatus 300 to apply error correction on the output of the classification. In an example embodiment, an error correction mechanism may include majority voting. It will be understood that certain other complex correlations may also be utilized for applying error correction to the classification output. In an embodiment, the processor 302 is configured to, with the content of the memory 304 , and optionally with other components described herein, to cause the apparatus 300 to compare the error in the sampled data with a threshold error. In an embodiment, the processor 302 is configured to, with the content of the memory 304 , and optionally with other components described herein, to cause the apparatus 300 to update or retain the sampling information based on a comparison of the error with a threshold error. In an embodiment, the threshold error may be preset in the apparatus 300 .
In an embodiment, the processor 302 is configured to, with the content of the memory 304 , and optionally with other components described herein, to cause the apparatus 300 to retain the sampling information by retaining values of the number of basis vectors and the basis size associated with the selection information on determination of the error to be lower than the threshold error.
In an embodiment, the processor 302 is configured to, with the content of the memory 304 , and optionally with other components described herein, to cause the apparatus 300 to update the sampling information by increasing the number of basis vectors associated with the subset of the complete basis matrix upon determination of the error to be greater than the threshold error.
In an embodiment, the processor 302 is configured to, with the content of the memory 304 , and optionally with other components described herein, to cause the apparatus 300 to update the sampling information by increasing the basis size associated with the subset of the complete basis matrix upon determination of the error to be greater than or equal to the threshold error during a threshold number of updating of the basis size. In an embodiment, the processor 302 is configured to, with the content of the memory 304 , and optionally with other components described herein, to cause the apparatus 300 to update the sampling information by reducing values of the number of basis vectors and the basis size associated with the selection information on determination of the error to be lesser than the threshold error.
In an example embodiment, the processor 302 is configured to, with the content of the memory 304 , and optionally with other components described herein, to cause the apparatus 300 to reclassify the activity on determination of the error to be greater than or equal to the threshold error. In an embodiment, the activity may be reclassified by utilizing the updated sampling information. In an example embodiment, a processing means may be configured to update or retain the sampling information based on a comparison of the error with a threshold error. An example of the processing means may include the processor 302 , which may be an example of the controller 208 .
FIG. 4 illustrates an apparatus 400 for activity recognition, in accordance with an example embodiment. The apparatus 400 for activity recognition may be employed, in a device that is configured to sense activity of a user. For example, the apparatus 400 may be employed in a variety of sensors. In an example embodiment, the apparatus 400 may be employed in wearable sensors that may be attached to a user's body to measure attributes of interest such as motion, location, temperature, electrocardiogram (ECG), and the like. Various embodiments may be embodied wholly at a single device or a combination of devices. It should also be noted that some of the devices or elements described below may not be mandatory and thus some may be omitted in certain embodiments.
In an embodiment, the apparatus 400 includes or otherwise is in communication with at least one processor 404 and at least one memory 406 . Examples of the at least one memory 406 include, but are not limited to, volatile and/or non-volatile memories. Some examples of the volatile memory include, but are not limited to, random access memory, dynamic random access memory, static random access memory, and the like. Some example of the non-volatile memory includes, but are not limited to, hard disks, magnetic tapes, optical disks, programmable read only memory, erasable programmable read only memory, electrically erasable programmable read only memory, flash memory, and the like. The memory 406 may be configured to store information, data, applications, instructions or the like for enabling the apparatus 400 to carry out various functions in accordance with various example embodiments. For example, the memory 406 may be configured to buffer input data comprising a sampling information for processing by the processor 404 . Additionally or alternatively, the memory 406 may be configured to store instructions for execution by the processor 404 .
The processor 404 may be embodied in a number of different ways. The processor 302 may be embodied as a multi-core processor, a single core processor; or combination of multi-core processors and single core processors. For example, the processor 404 may be embodied as one or more of various processing means such as a coprocessor, a microprocessor, a controller, a digital signal processor (DSP), processing circuitry with or without an accompanying DSP, or various other processing devices including integrated circuits such as, for example, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a microcontroller unit (MCU), a hardware accelerator, a special-purpose computer chip, or the like. In an example embodiment, the multi-core processor may be configured to execute instructions stored in the memory 406 or otherwise accessible to the processor 404 . Alternatively or additionally, the processor 404 may be configured to execute hard coded functionality. As such, whether configured by hardware or software methods, or by a combination thereof, the processor 404 may represent an entity, for example, physically embodied in circuitry, capable of performing operations according to various embodiments while configured accordingly. For example, if the processor 404 is embodied as two or more of an ASIC, FPGA or the like, the processor 404 may be specifically configured hardware for conducting the operations described herein. Alternatively, as another example, if the processor 404 is embodied as an executor of software instructions, the instructions may specifically configure the processor 404 to perform the algorithms and/or operations described herein when the instructions are executed. However, in some cases, the processor 404 may be a processor of a specific device, for example, a mobile terminal or network device adapted for employing embodiments by further configuration of the processor 404 by instructions for performing the algorithms and/or operations described herein. The processor 404 may include, among other things, a clock, an arithmetic logic unit (ALU) and logic gates configured to support operation of the processor 404 .
In an embodiment, one or more sensors employing the apparatus 400 may be configured to sense human activity. In an embodiment, in addition to the processor 404 and memory 406 , the apparatus 400 may include a sensing element 402 , a wireless link module 4141 , and an activity class module 416 . In an embodiment, the processor may further include an analog to digital converter 410 , a random number generator 408 and a projection sampling module 412 . In an embodiment, the sensing element 402 is configured to sense the activity. In an embodiment, the A/D converter 410 is coupled with the sensing element 402 , and is configured to perform an analog to digital conversion of the activity data sensed by the sensing element 402 . In an embodiment, an output of the A/D converter 410 may be utilized for performing sampling of the activity data. In an embodiment, the output of the A/D converter may be stored in a memory, for example the memory 406 of the apparatus 400 . In an embodiment, the output of the A/D converter 410 may be stored in the memory 406 until a data vector associated with the activity data of length N is obtained.
As discussed above, the activity data may be converted from analog domain to digital domain for performing random sampling. Alternately, random sampling may be performed in the analog domain, for example, by using programmable amplifier and an integrator circuit.
In an embodiment, the processor 404 is configured to, with the content of the memory 406 , and optionally with other components described herein, to cause the apparatus 400 to performing random sampling of the activity data based on a sampling information sent by a device, for example, the device 200 (explained with reference to FIG. 2 ). In an embodiment, the processor 404 is configured to, with the content of the memory 406 , and optionally with other components described herein, to cause the apparatus 400 to receive a sampling information from the apparatus 300 . In an embodiment, the sampling information includes a basis matrix and a selection information for selecting at least a subset of the basis matrix. In an embodiment, the subset of basis matrix may include a set of randomly selected basis vectors along which the activity data may be sampled to generate the sampled data. In an embodiment, the basis matrix may be a complete basis matrix wherein the basis vectors may span an entire predefined space as opposed to a subspace of the predefined subspace. In an embodiment, the complete basis matrix associated with the sampling information may be an N×N matrix. In an embodiment, a processing means may be configured to receive a sampling information from the apparatus 300 . An example of the processing means may include the processor 404 .
In an embodiment, the processor 404 is further configured to, with the content of the memory 406 , and optionally with other components described herein, to cause the apparatus 400 to store the sampling information received from the apparatus 300 . In an embodiment, a memory means may be configured to store the sampling information received from the apparatus 300 . An example of the memory means may include the memory 406 .
In an embodiment, the processor 404 is configured to, with the content of the memory 406 , and optionally with other components described herein, to cause the apparatus 400 to perform compressive sampling of an activity data associated with the activity based at least on the sampling information and a sampling criteria to generate a sampled data associated with the activity. In an embodiment, the sampling criteria includes selection of an incoherent basis for performing compressive sampling of the activity data.
In an embodiment, the processor 404 is configured to, with the content of the memory 406 , and optionally with other components described herein, to cause the apparatus 400 to select the subset of the basis matrix based on the selection information. For example, a subset of M basis vectors may be randomly selected from the basis matrix (of size N×N) to form an M×N sampling matrix. The M×N sampling matrix may be denoted by ‘A’. In an example embodiment, a processing means may be configured to randomly select the set of basis vectors based on the sampling information. An example of the processing means may include processing means, for example a random number generator 408 .
In an embodiment, the processor 404 is configured to, with the content of the memory 406 , and optionally with other components described herein, to cause the apparatus 400 to compressively sample the activity data along the set of randomly selected basis vectors and in response generate a sampled data. In an example embodiment, for sampling the activity data along the set of basis vectors, the processor 404 is configured to, with the content of the memory 406 , and optionally with other components described herein, to cause the apparatus 400 to multiply the data vector (x) with a sampling matrix A to obtain a sample vector (y) of length M. In an embodiment, a processing means may be configured to multiply the data vector with the sampling matrix A to obtain the sample vector of length M. An example of the processing means may include a projection sampling module 412 . The sampled data may be generated based on the expression:
y=A T x, where, A denotes the N×M sampling matrix whose columns are the (orthogonal) basis vectors, and x denotes the N dimensional uncompressed data vector
In an embodiment, the processor 404 is configured to, with the content of the memory 406 , and optionally with other components described herein, to cause the apparatus 400 to send the sampled data to the apparatus (for example the apparatus 300 ) for facilitating labeling of the activity. In an embodiment, the apparatus 400 may send the sampled data to the apparatus 300 until a frame of the sampled data may be sent to the apparatus 300 . In an example embodiment, the frame size may be about 0.1 s-10s. In an embodiment, the sampled data may be sent to the apparatus 300 through the wireless link module 414 .
In an embodiment, the apparatus 400 may send the sampled data to the apparatus 300 . A mode of operation of the apparatus 400 wherein the apparatus is configured to send only the sampled data may be referred to as ‘unlabeled mode’. In another embodiment, the apparatus 400 may send a user-selected label to the apparatus 300 . In an embodiment, the apparatus 400 may include a user interface for receiving a user input associated with the label from the user. For example, the user may input the label by utilizing the activity class module 416 . The activity class module 416 may be coupled with the wireless link module 414 for facilitating sending of the activity class provided by the user to the apparatus 200 . As disclosed herein, a mode of operation of the apparatus 400 wherein the apparatus is configured to send a label along with the sampled data to the device 300 may be referred to as ‘labeled mode’.
As already discussed with reference to FIG. 3 , the apparatus 300 may utilize the sampled data for classifying the label to be assigned to the activity. Additionally, the apparatus 300 may determine an error in the sampled data. In an embodiment, when the error in the sampled data is determined to be greater than a threshold error, then the apparatus 300 is configured to update the selection information and resend the updated selection information to the apparatus 400 .
In an embodiment, the processor 404 is configured to, with the content of the memory 406 , and optionally with other components described herein, to cause the apparatus 400 to receive the updated sampling information from the apparatus 300 . In an embodiment, the processor 404 is configured to, with the content of the memory 406 , and optionally with other components described herein, to cause the apparatus 400 to perform compressive sampling of the activity data based at least on the updated selection information to generate an updated sampled data. In an embodiment, the updated sampled data may be generated in a manner similar to the generation of the sampled data. In an embodiment, the processor 404 is configured to, with the content of the memory 406 , and optionally with other components described herein, to cause the apparatus 400 to send the updated sampled data to the apparatus 300 for facilitating relabeling of the activity. In an embodiment, the wireless link module 410 may be configured to send the updated sampled data to the apparatus 300 .
As already discussed, in an embodiment, apparatus 400 may include the sensing element 402 , the processor 404 , the memory 406 , the wireless link module 414 and the activity class module 416 . In an embodiment, the processor 404 may include the A/D converter 410 , the random number generator 408 , and the projection sampling module 412 . In an alternate embodiment, instead of separate modules mentioned above, the apparatus 400 may include an integrated hardware solution implementing the modules for A/D converter, sensing element, random number generator, and projection sampling module, thereby precluding the need of separate modules. For example, the apparatus 400 may include smart sensors in which analog or mixed signal circuitry are combined with the sensor readout electronics.
FIG. 5 illustrates an example of compressive sampling of an activity data for activity recognition, in accordance with an example embodiment. In an example embodiment, for performing compressive sampling, an analog to digital conversion of an activity data 502 is performed and a data vector 504 associated the activity data 502 is generated. In an embodiment, the data vector 504 may be sampled along randomly selected basis vectors.
In an embodiment, a data vector such as a data vector 504 of length N (e.g. in time domain) may be projected along a set of basis vectors (for example basis vectors 506 , 508 ). In an embodiment, the set of basis vectors may be randomly selected based on a sampling information received from the apparatus 300 . In another embodiment, the set of basis vectors may be selected randomly and orthonormalised. In an example embodiment, the set of basis vectors may be orthonormalised based on a Gram-Schmidt procedure.
In an example embodiment illustrated in FIG. 5 , the set of randomly selected basis vectors 506 , 508 associated with the sampling matrix may be multiplied with the data vector 504 to obtain samples such as samples, 510 , 512 . Let x denote the N dimensional uncompressed data vector and let A denote an N by M sampling matrix whose columns are the (orthogonal) basis vectors. Then, the sampled data is an M dimensional vector y such that:
y=A T x.
In an example embodiment, the basis vectors such as basis vectors 506 , 508 may include various non-zero real valued entries. In such a scenario, the samples may be obtained by summing up the components. In an example embodiment, column vectors may be utilized as sampling vectors.
FIG. 6 illustrates a sequence of transmission of sampled data in accordance with an embodiment. As previously discussed, the sampled data 610 (represented by y) may be transmitted or sent to the apparatus 300 for facilitating learning and activity recognition. In an embodiment, activity recognition may be performed based on the sampled data received from multiple sensors that may be configured to sense different quantities at different sampling rates. For example, sensors employed in an accelerometer and compass may facilitate in sensing of accelerometer data and compass orientation data, respectively. The accelerometer data and the compass orientation data may be sensed and subsequently sampled by different sensors. In an embodiment, the activity data received from multiple vectors y, (for example, 612 , 614 , 616 , 618 ) may generate a data set that may be represented as:
{y i ,i=1, . . . , M}
In an embodiment, the activity data received from the multiple sensors may be arranged into a vector prior to performing random projection. For example, the data associated with the data set 610 may be arranged in a vector 620 prior to performing random projection of the activity data. For example, as illustrated in FIG. 6 , the data from the vectors 612 , 614 , 616 may be received and stored at location 622 , 624 , 626 respectively of the vector 620 . In an embodiment, the data may be arranged in a vector 620 in a sequential manner, i.e. in sequence of receiving of the data from the respective sensor at the apparatus 300 .
FIG. 7 is a flowchart depicting an example method 700 for activity recognition, in accordance with an example embodiment. The method 700 depicted in the flow chart may be executed by, for example, the apparatus 300 of FIG. 3 . In an embodiment, the method may be executed by a device that may be communicably coupled to one or more sensors. In an embodiment, the device may be a mobile phone, a PDA, a laptop and the like.
In an embodiment, the apparatus 300 may be configured to recognize an “activity” being performed by a user based on a sampled data received from one or more sensor. In an embodiment, the activity may include walking, jogging, running, and any other motion or movement phenomenon being performed by the user. In an embodiment, the activity may be sensed by the one or more sensors. For example, the one or more sensors may capture an activity data associated with the activity, and process the activity data to generate sampled data. In an embodiment, the sampled data may be generated by the one or more sensors based on a sampling information that may be sent by the apparatus 300 to the one or more sensors. In an embodiment, the sampling information includes a basis matrix and a selection information for selecting at least a subset of the basis matrix. In an embodiment, the basis matrix may be a complete basis matrix wherein the basis vectors may span an entire predefined space as opposed to a subspace of the predefined subspace.
At block 702 , the method 700 includes receiving the sampled data associated with the activity at the apparatus 300 from the one or more sensors. In an embodiment, the sampled data may be sent to the apparatus 300 by one or more devices embodying the apparatus 400 (explained in FIG. 4 ). In an embodiment, examples of the device embodying the apparatus 400 may include sensors such as an accelerometer, a barometer, a compass, and the like. At block 704 , the activity may be classified based at least on the sampled data. In an embodiment, classifying the activity may include assigning a label to the activity. In an embodiment, the activity may be classified based on a classification rule or model. For example, the sampled data may be classified based on a quadratic discriminant analysis. In another embodiment, the sampled data may be classified based on the classification rule as well as a user input. In an embodiment, the user input may be sent by the one or more sensors.
In an embodiment, the sampled data received at the device may include error due to noise associated with the one or more sensors. In an embodiment, the error in the sampled data may lead to a wrong classification of the activity. At block 706 , an error in the classification of the activity may be determined. In an embodiment, a variance in an output of the classification of the activity may be utilized as a measure for error determination since user activities may remain substantially same (i.e. do not fluctuate) over a very short time scale (seconds). Accordingly, at different time instances during this short time interval, the sampled data may be computed a threshold number of times, and classified by assigning a label thereto. In case, a variation in the classification of activity is detected during the classification, the classification of the activity may be regarded as incorrect.
In an embodiment, the error in the classification may be computed and compared with a threshold error. In an embodiment, the threshold error may be preset in the apparatus 400 . In an embodiment, based on a comparison of the error with the threshold error, the sampling information may be updated or retained at block 708 . For example, on detection of absence of any errors in classification, the number of vectors associated with the sampling information may be retained or reduced. In an embodiment, the size of the sampling matrix may be reduced by reducing the number of vectors therein. In an alternate embodiment, the size of the sampling matrix may be retained as such if the size of the sampling matrix is determined to be already at a minimum allowable size. In an embodiment, the minimum allowable size of the matrix may be preset in the apparatus 400 .
In an embodiment, on determination of error being greater than or equal to the threshold, an error correction may be applied to the sampled data. Examples of error correction methods includes, but are not limited to majority voting, and the like. In an example embodiment, on determination of error being greater than or equal to the threshold error, the sampling information may be updated. For example, based on the determined error to be greater than the threshold error, the number of basis vectors may be increased. Additionally or optionally, if the error in the classification performed during previous predetermined number of times is determined to be greater than the threshold error, then the size of the complete basis matrix associated with the sampling information may be increased. For example, if the error is determined to be greater than or equal to the threshold error during last ten classifications, then the size of the complete basis matrix may be increased.
In an example embodiment, a processing means may be configured to perform some or all of: receiving, at a device, a sampled data associated with an activity from one or more sensors wirelessly coupled to the device, the sampled data being generated at the one or more sensors based on a compressive sampling of an activity data associated with the activity, the compressive sampling being performed based on a sampling information; classifying the activity based at least on the sampled data; determining an error associated with the classification of the activity; and updating the sampling information based on a comparison of the error with a threshold error. An example of the processing means may include the processor 302 , which may be an example of the controller 208 .
FIG. 8 is a flowchart depicting an example method 800 for activity recognition, in accordance with an example embodiment. The method 800 depicted in the flow chart may be executed by, for example, the apparatus 400 of FIG. 4 . For example, the method may executed by wearable sensors comprising the apparatus 400 . The sensors may be attached to a user's body to measure an activity data associated with a user activity. In an embodiment, the activity data may include attributes such as motion, location, temperature, ECG, and the like. In an embodiment, examples of devices embodying the apparatus 400 may include sensors such as an accelerometer, a barometer, a compass, and the like.
At block 802 , the method includes receiving a sampling information at an apparatus (for example, the apparatus 400 , explained in FIG. 4 ) from another apparatus (for example, the apparatus 300 , explained in FIG. 3 ) wirelessly coupled to the apparatus 400 . Herein, apparatus 400 may be embodied in a sensor. Accordingly, the description of method 800 may be explained by referring apparatus 400 as sensor, and vice-versa. In an embodiment, the sampling information includes a basis matrix and a selection information for selecting at least a subset of the basis matrix. In an embodiment, the basis matrix may be a complete basis matrix wherein the basis vectors may span an entire predefined space as opposed to a subspace of the predefined subspace. In an embodiment, the complete basis matrix associated with the sampling information may be an N×N matrix.
At block 804 , compressive sampling of the activity data may be performed based at least on the sampling information and a sampling criteria. In an embodiment, the compressive sensing may be performed to generate a sampled data associated with the activity. In an embodiment, the sampling criteria may include selection of an incoherent basis for performing compressive sampling. In an embodiment, the subset the complete basis matrix may be selected based on the selection information. For example, a set of M basis vectors may be selected from the complete basis matrix to form an M×N sampling matrix. The M×N sampling matrix may be denoted by ‘A’. In an embodiment, the sampled data may be generated by performing compressive sampling of the activity data along with the set of basis vectors that may be randomly selected based on the sampling information. For example, a data vector (x) of length N (e.g. in time domain) may be projected along the set of basis vectors. The sampled data may be generated based on the expression:
y=A T x, where, A denotes the N×M sampling matrix whose columns are the (orthogonal) basis vectors, and x denotes the N dimensional uncompressed data vector
At block 806 , the sampled data may be sent to the apparatus (for example, the apparatus 300 ) for facilitating labeling of the activity. In an embodiment, the sampled data may be sent to the apparatus until a frame of the sampled data may be received at the apparatus. In an example embodiment, the frame size may be about 0.1 s-10s. In an embodiment, the sampled data may be sent to the apparatus 300 over a wireless link.
FIG. 9 is a flowchart depicting an example method 900 for activity recognition, in accordance with an example embodiment. The method 900 depicted in the flow chart may be executed by, for example, the apparatuses 300 and 400 of FIGS. 3 and 4 , respectively. Operations of the flowchart, and combinations of operation in the flowchart, may be implemented by various means, such as hardware, firmware, processor, circuitry and/or other device associated with execution of software including one or more computer program instructions. For example, one or more of the procedures described in various embodiments may be embodied by computer program instructions. In an example embodiment, the computer program instructions, which embody the procedures, described in various embodiments may be stored by at least one memory device of an apparatus and executed by at least one processor in the apparatus. Any such computer program instructions may be loaded onto a computer or other programmable apparatus (for example, hardware) to produce a machine, such that the resulting computer or other programmable apparatus embody means for implementing the operations specified in the flowchart. These computer program instructions may also be stored in a computer-readable storage memory (as opposed to a transmission medium such as a carrier wave or electromagnetic signal) that may direct a computer or other programmable apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture the execution of which implements the operations specified in the flowchart. The computer program instructions may also be loaded onto a computer or other programmable apparatus to cause a series of operations to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions, which execute on the computer or other programmable apparatus provide operations for implementing the operations in the flowchart. The operations of the method 900 are described with help of apparatus 300 and 400 of FIGS. 3 and 4 , respectively. However, the operations of the method can be described and/or practiced by using any other apparatus.
The method 900 provides steps for activity recognition in context-awareness based applications. Examples of activities that may be utilized for activity recognition may include walking, jogging, running, and the like. In an embodiment, one or more sensors may acquire and process data associated with user activities. In an embodiment, one or more sensors may be embodied in one or more wearable devices that may be attached to a user's body so as to acquire activity data by measuring various attributes associated with activity recognition. For example, the one or more sensors may be configured to measure motion, location, temperature, humidity, audio level and other such attributes required for activity recognition. In an embodiment, the one or more sensors may process the acquired data and send the processed activity data to the apparatus 300 . In an embodiment, the apparatus 300 may be embodied in a device, for example the device 200 (explained with reference to FIG. 2 ). The device 200 may utilize the processed activity data to classify the “activity” of the user. An information regarding the user's activity may be utilized to provide activity-dependent user experiences.
At block 902 of method 900 , a basis matrix associated with a sampling information may be computed at the device, for example the device 200 . In an embodiment, the sampling information may include the basis matrix and a selection information for selecting at least a subset of the basis matrix. In an embodiment, the sampling information may be stored, for example in a memory of the apparatus 300 . At block 904 , the sampling information may be sent to the one or more apparatus, for example the apparatus 400 (explained with reference to FIG. 3 ). In an embodiment, the one or more apparatus 300 may include one or more sensors. In an embodiment, the one or more apparatus 300 may be communicatively coupled with the device 200 . In an embodiment, the one or more sensors may be coupled with the apparatus 300 through a wireless link.
At block 906 , the one or more sensors may receive the basis matrix. At block 908 , the one or more sensors may store the basis matrix. In an embodiment, the one or more sensors may store the basis matrix in a memory, for example the memory 406 (refer to FIG. 4 ).
At block 910 , the device 200 may send a selection information to the one or more sensors for selecting a subset of the basis matrix. In an embodiment, the selection information includes instructions for selecting the subset of the complete basis matrix. In an embodiment, the instructions may include one or more basis vectors and a basis size that may be selected from the complete basis matrix.
At block 912 , a sampling of an activity data associated with a user activity is performed. In an embodiment, the sampling of the activity data is performed at the one or more sensors. In an embodiment, the sampling of the activity data is performed based at least on the complete basis matrix and the selection information received from the device 200 . In an embodiment, sampling may be performed based on compressive sampling by utilizing an incoherent basis.
In an embodiment, the activity data may be sampled along with the set of basis vectors that may be randomly selected based on the sampling information. A data vector (denoted by x) of length N (e.g. in time domain) may be projected along the set of basis vectors. The projection of the data vector along the set of randomly selected basis vectors is explained with reference to FIG. 5 . Based on sampling, the sampled data may be generated as per the following expression:
y=A T x, where, A denotes an N×M sampling matrix whose columns are the (orthogonal) basis vectors, and x denote the N dimensional uncompressed data vector associated with the activity data.
At block 914 , the sampled data may be sent by the one or more sensors to the device. In an embodiment, the sampled data may facilitate in labeling of the activity at the device. At block 916 , the sampled data sent by the one or more sensors may be received at the device. At block 918 , the activity may be classified based at least on the sampled data and a classification rule. In an embodiment, the classification of the activity may include assigning a label to the activity. In an embodiment, the classification rule may include a model comprising a training set of labeled samples. In an example embodiment, the training set with labels may be represented in the basis that is utilized for sampling of the activity data at the one or more sensors. In another example embodiment, the labels in the training set may be provided initially (e.g. as in “factory settings”) in a complete basis from which it is easy to calculate the set in any undercomplete basis such as the one used for compressive sampling of the activity data. In another embodiment, the labels associated with the training set may be deduced from other sensory data. In yet another example embodiment, the labels may be obtained based on an unsupervised learning. For example, various identify clusters (e.g. k-means algorithm) may be generated and labels may be obtained by “asking” information from the user.
In an embodiment, the sampled data output from the one or more sensors may include error due to noise or minute perturbations in the activity data observed by the one or more sensors. Since the activities performed by the users do not undergo substantial fluctuation over a very short time scale (for example, seconds), a variance in the output of the classification output may be utilized for detection of any errors in the classification. At block 920 , an error in the sampled data may be determined. In an example embodiment, an error correction may be applied to the output of the classification. In an example embodiment, an error correction mechanism may include majority voting. It will be understood that certain other complex correlations may also be utilized for applying error correction to the classification output.
At block 922 , it may be determined whether is error in the classification of the activity is greater than a threshold error. In an embodiment, the threshold error may be fixed prior to performing sampling of the activity data. In an embodiment, on detection of absence of any errors in classification, the number of vectors associated with the sampling information may be retained or reduced. In an embodiment, the size of the sampling matrix may be reduced by reducing the number of vectors therein. In an alternate embodiment, the size of the sampling matrix may be retained as such as block 924 , if the size of the sampling matrix is determined to be already at a minimum allowable size. In an embodiment, the minimum allowable size of the matrix may be preset in the device.
In an embodiment, on determination of error being greater than or equal to the threshold error, the compressive sampling metrics may be updated at block 926 . For example, based on the determined number of error to be greater than the threshold error, the number of vectors may be increased. Additionally or optionally, if the error associated with previous predetermined number of times is determined to be greater than the threshold error, then the size of the basis matrix associated with the sampling information may be increased.
At block 928 , the updated selection information may be sent to the one or more sensors for selecting at least a portion of the sampling information. In an embodiment, the at least the portion of the sampling information selected based on the updated selection information is different from the subset of sampling information that was previously utilized for generating the sampled data. In an embodiment, the at least a portion of the sampling information selected based on the updated selection may be utilized for reclassifying the activity.
To facilitate discussion of the methods 700 , 800 and/or 900 of FIGS. 7 , 8 and 9 , certain operations are described herein as constituting distinct steps performed in a certain order. Such implementations are exemplary and non-limiting. Certain operation may be grouped together and performed in a single operation, and certain operations can be performed in an order that differs from the order employed in the examples set forth herein. Moreover, certain operations of the methods 700 , 800 and/or 900 are performed in an automated fashion. These operations involve substantially no interaction with the user. Other operations of the methods 700 , 800 and/or 900 may be performed by in a manual fashion or semi-automatic fashion. These operations involve interaction with the user via one or more user interface presentations.
Without in any way limiting the scope, interpretation, or application of the claims appearing below, a technical effect of one or more of the example embodiments disclosed herein is to perform activity recognition. As explained in FIGS. 1-9 , The embodiments disclosed herein provide methods and devices for performing activity recognition. In various embodiments, the devices such as distributed sensors may be configured to acquire data and process the acquired data. The acquired data may be processed based on compressive sampling. The processed data (or the sampled data) may be sent wirelessly to a device (for example a smartphone). The device utilizes the sampled data directly for classifying the “activity” of the user. In an embodiment, the information about the activity can then be used to provide activity-dependent user experiences. The disclosed method and devices are advantageous since the sampled data may be utilized directly for performing activity recognition, thereby precluding a need for decompression. Additionally, since the sampled data may be directly utilized for activity recognition, the disclosed methods and devices facilitates low bandwidth requirement and a lower power consumption. Also, since the data is compressively sampled at the wireless devices, it can be economically transmitted and stored on the central device. Further, using “random” sampling provides a degree of privacy as it may not be easy to deduce everything from the data without knowing the sampling basis. The information about the basis to be used may only be stored on the user's device or other secure storage. In an embodiment, the classification may be performed even below the decompression threshold. Moreover, the activity data automatically undergoes a dimensionality reduction step which is beneficial for classification/machine learning.
Various embodiments described above may be implemented in software, hardware, application logic or a combination of software, hardware and application logic. The software, application logic and/or hardware may reside on at least one memory, at least one processor, an apparatus or, a computer program product. In an example embodiment, the application logic, software or an instruction set is maintained on any one of various conventional computer-readable media. In the context of this document, a “computer-readable medium” may be any media or means that can contain, store, communicate, propagate or transport the instructions for use by or in connection with an instruction execution system, apparatus, or device, such as a computer, with one example of an apparatus described and depicted in FIGS. 3 and/or 4 . A computer-readable medium may comprise a computer-readable storage medium that may be any media or means that can contain or store the instructions for use by or in connection with an instruction execution system, apparatus, or device, such as a computer. In one example embodiment, the computer readable medium may be non-transitory.
If desired, the different functions discussed herein may be performed in a different order and/or concurrently with each other. Furthermore, if desired, one or more of the above-described functions may be optional or may be combined.
Although various aspects of the embodiments are set out in the independent claims, other aspects comprise other combinations of features from the described embodiments and/or the dependent claims with the features of the independent claims, and not solely the combinations explicitly set out in the claims.
It is also noted herein that while the above describes example embodiments of the invention, these descriptions should not be viewed in a limiting sense. Rather, there are several variations and modifications, which may be made without departing from the scope of the present disclosure as defined in the appended claims.