(en)The present invention is directed to a “smart earplug” capable of selectively adjusting the output of an array of acoustic wave generation elements or speakers within a user's ear canal in response to input signals, wherein at least one of the input signals has been at least partially attenuated.

1.ApplicationNumber: US-78232107-A
1.PublishNumber: US-2008044040-A1
2.Date Publish: 20080221
3.Inventor: WERNER ALAN J.JR.
BUGLER MALCOLM

4.Inventor Harmonized: WERNER ALAN J JR(US)
BUGLER MALCOLM(US)

5.Country: US
6.Claims:
(en)The present invention is directed to a “smart earplug” capable of selectively adjusting the output of an array of acoustic wave generation elements or speakers within a user's ear canal in response to input signals, wherein at least one of the input signals has been at least partially attenuated.

7.Description:
(en)This application is a Continuation-in-Part of U.S. application Ser. No. 11/254,448 for a “METHOD AND APPARATUS FOR INTELLIGENT ACOUSTIC SIGNAL PROCESSING IN ACCORDANCE WITH A USER PREFERENCE,” filed Oct. 20, 2005, and claims priority from U.S. Provisional Application 60/621,560 by Alan J. Werner for a “METHOD AND APPARATUS FOR INTELLIGENT ACOUSTIC SIGNAL PROCESSING IN ACCORDANCE WITH A USER PREFERENCE,” filed Oct. 22, 2004, and the present application further claims priority from U.S. Provisional Application 60/820,178 by Alan J. Werner for a “METHOD AND APPARATUS FOR INTELLIGENT ACOUSTIC SIGNAL PROCESSING IN ACCORDANCE WITH A USER PREFERENCE,” filed Jul. 24, 2006, all of which are hereby incorporated by reference in their entirety.
The present invention is directed to an acoustic control apparatus and method, and more particularly to a “smart earplug” that is capable of selectively adjusting the output of an array of acoustic wave generation elements in response to input signals, wherein at least one of the input signals has been at least partially attenuated.


BACKGROUND AND SUMMARY
The following patent is noted and the teachings thereof are hereby incorporated by reference: U.S. Pat. No. 6,768,803, to Duhamel, issued Jul. 27, 2004 for a “METHOD AND APPARATUS FOR SELECTIVE ACOUSTIC SIGNAL FILTERING.”
It is well known that noise in the work place can both mask important audio “information” and cause permanent physical damage to the human “hearing” system. For example, in the heavy construction industry, the wearing of sound blocking “ear muffs” is a common solution. The problem also exists in the performing arts arena, particularly in the “loud” jazz and heavy rock music communities. The normal solution is to use earplugs, which are small, rubbery or foam devices that are inserted into the ear channel to “block” the sound. However, such devices tend to block out or attenuate all of the acoustic signals, thereby reducing or eliminating certain signals to a level where they cannot be heard or appreciated by the listener.
A similar problem exists in the classical music industry as James R. Oestrich suggests in an article published on Jan. 11, 2004 in the New York Times. Auditory acuity and sensitivity are especially important to the musician and even a subtle hearing deficit may detract from the musician's performance, and in extreme cases, severe hearing loss could mean an end to a performing career. The Occupational Safety and Health Administration (OSHA) regulates noise levels in the workplace and exposure levels within the Orchestra may not exceed an 8 hour Time Weighted Average (TWA) of 90 dBA. The ceiling level is 115 dBA, meaning that and at no time may this level be exceeded 115 dBA. Exposure to excessive sound levels can cause damage in two ways: mechanical trauma and sensorineural hearing loss. Sensorineural hearing loss, caused by repeated exposure to excessive noise levels, is of most concern to musicians. In addition to the auditory effects, noise can cause physiological and/or psychological problems as well. The physiological effects may include a wide variety of symptoms including increased heart rate, blood pressure, breathing rate, muscle contractions and perspiration. Psychological complaints may include nervousness, tension, anger and irritability.
However, this problem has not been addressed due to the inherent limitations of conventional hearing aids or ear plugs, including lack of control as to the amount of attenuation desired by a listener, as well as a control over the directionality of the attenuation. In other words, performers may wish to more heavily attenuate the percussion or brass section behind them, but to keep the woodwinds to the side or strings in front of them at a higher or non-attenuated level. In any live musical performance, it is critical to “hear” exactly what is going on around you. This may be for better balance, a matching of tonal quality, a “clue” as to when to play, etc. Not only is this “audio information” important, but so is the location or direction from which it is coming.
As mentioned above, the problem with the conventional ear muff and ear plug approaches is that not only is the quality of the sound changed, but that any directionality is lost. The present invention is, therefore, directed to an improved or “smart” ear plug (in the ear or not), that provides true acoustic rendition of the sound, wherein the amplitude or similar signal characteristics of the acoustic signal may be controlled on a directional basis.
The advent of micro-electronics provides new options for the sensing and delivery of acoustic information or signals. Micro-electronics makes physically small circuitry and electromechanical systems possible. In accordance with one aspect of the present invention, there is provided an array of very small micro-electromechanical systems (MEMS) microphones to detect the acoustic waves or vibrations coming from a plurality of directions (e.g., front/rear, left/right side, above, below, etc.). Having received the various, discrete signals from the array of MEMS microphones, with their inherent directionality; a similar array of MEMS speakers or “audio transducers”, could be used to generate the output (perhaps conditioned to attenuate the signal from certain directions more than other directions). Thus, the system would provide a user with all of the audio information, but with selective attenuation (or gain) based upon directionality of the acoustic source—providing the impression of being from the same direction with the same audio information but at a user adjustable sound level.
In accordance with the present invention, there is provided an acoustic control apparatus, comprising: input sensor for receiving an input acoustic signal to be processed, said input sensor including a microphone array, said microphone array manifesting vibration in response to interaction with the input acoustic signals from a plurality directions to generate a plurality of input signals, each representing an acoustic input from one of the plurality of directions relative to said input sensor; a signal processing device for producing, in response to the input signals, at least one output signal, said signal processing device characterized by a uniform frequency response such that the output acoustic signal spectrum level is generally reflective of an input acoustic signal spectrum level, said signal processing device further including a mixing circuit to enable a mixing of at least two acoustic signals from the plurality of directions; and an acoustic output port for generating the output acoustic signal produced by said signal processing device.
In accordance with another aspect of the invention, there is provided an acoustic signal processing system for processing acoustic signals in accordance with a user preference, comprising: at least one microphone array, said microphone array generating a plurality of input signals in response to acoustic vibrations, each input signal representing an acoustic input from one of a plurality of directions relative to said microphone array; a signal processing device for producing, in response to the plurality of input signals, at least one output signal, said signal processing device characterized by a uniform frequency response such that an output acoustic signal spectrum level is generally reflective of an input acoustic signal spectrum level, said signal processing device further including a mixing circuit with a crossover network, responsive to the user preference, to mix at least two acoustic signals from the plurality of directions; and at least one speaker for generating the output acoustic signal in response to the output signal from said signal processing device.
In accordance with a further aspect of the invention, there is provided a method for controlling the sound perceived by a user, comprising: receiving, using a micro-electronic microphone array, an input acoustic signal and generating a plurality of input signals representing the acoustic input from each of a plurality of directions relative to the array; processing the input signals to produce at least one output signal such that the output signal spectrum level is generally reflective of an input acoustic signal spectrum level, including mixing of at least two acoustic signals from the plurality of directions to produce the at least one output signal; and generating, by an output speaker responsive to the at least one output signal an acoustic signal directly in the canal of a user's ear.
The techniques described herein are advantageous because they provide a reduced-size method of controlling the audio or acoustic input received by a user, thereby enabling a user to function in an acoustically unfriendly environment without the complete loss or exclusion of acoustic information. The techniques of the invention are advantageous because they provide a range of alternatives, each of which is useful in appropriate situations. As a result of the invention, it is anticipated that musicians, construction workers and the like may find improved on-the-job experience and reduced hearing loss due to loud noises.


BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is an exemplary assembly-type illustration of an embodiment of the present invention;
FIG. 2 is an illustrative embodiment of the invention, wherein user controls are worn on a necklace by a musician;
FIGS. 3 and 4 A- 4 B is another embodiment of the invention, wherein the user controls and power supply are located on a belt for attachment to a user;
FIG. 5 is an illustration of a circuit that may be employed to process the acoustic signals in accordance with an aspect of the invention; and
FIG. 6 is an exemplary block diagram of an embodiment of the invention as depicted in FIG. 5 .


The present invention will be described in connection with a preferred embodiment, however, it will be understood that there is no intent to limit the invention to the embodiment described. On the contrary, the intent is to cover all alternatives, modifications, and equivalents as may be included within the spirit and scope of the invention as defined by the appended claims.
DETAILED DESCRIPTION
For a general understanding of the present invention, reference is made to the drawings. In the drawings, like reference numerals have been used throughout to designate identical elements. The drawings are not to scale and some portions thereof may be depicted in a disproportionate fashion in order to depict detail.
The invention is directed to a “smart ear plug” that is dynamically controllable by a user. In one embodiment the device uses a system of four very small microphones, an “in ear” headset and a user worn control panel. It is battery powered. The microphone “set” consists of two microphone pairs. Each pair has one microphone aimed to the back of the user and one microphone aimed to the front of the user. One pair is for the left ear and the other is for the right ear. Full frequency response is maintained (20 Hz to 20 KHz) to maintain the “quality” of the sound. It is further contemplated that one or more of the microphones in the microphone set includes an amplifier to amplify the signal produced by the microphone (on-board), before the signal is transferred to other components for further processing, etc.
Set up is simple: the user adjusts the signal from the front microphones so that the user hears no difference using the system or not using the system. Thus the sound level of the persons adjacent to the user is the same with the system on or off. This is critical between musicians who must balance with each other such as the musicians immediately next to them. Critical examples are the principal and second of a section and principals of adjacent sections such as oboe, flute and clarinet. The levels from the rear where the inherently loud instruments such heavy brass and percussion are located are then reduced to a safe level for the user. There is also a left-right balance basically useful for the string sections.
Referring to FIG. 1 , depicted therein is an exemplary embodiment of the present invention for an acoustic control/filtering apparatus suitable for use by a classical musician or the like. In one embodiment, the apparatus comprises an input sensor 20 for receiving an input acoustic signal to be processed. Sensor 20 includes an array of directionally oriented microphones 24 A, 24 B for sensing sound from locations A and B, respectively. As depicted in the figure, microphones 24 A and 24 B might be employed to sense the directionality of acoustic vibrations coming say from front (A) and rear (B) positions relative to the user's ear 18 . It will be appreciated that the input sensor such as microphone array 20 will serve to manifest the acoustic vibration and thereby generate a plurality of input signals (at least A and B), each representing an acoustic input from one of a plurality of directions relative to said sensor 20 .
Although not depicted, it will be appreciated that various configurations for sensor 20 may be employed, including additional microphones 24 . It will also be appreciated that the directionality of the various microphones in the array may be enhanced through the use of baffles or similar means for isolating or separating the microphones in the array. As depicted, for example, in FIGS. 4A and 4B , the microphones may be mounted on the opposite sides a printed circuit board 23 . Although described relative to a microphone array, the present invention may be produced with only a single microphone operating for each ear of the user, so that the term microphone array may include one or more microphones. The same would be true, of course, for speakers employed in each ear of the user—where one or more speakers may be used in each ear.
In one embodiment, the microphones 24 are preferably micro-electronic or MEMS-type devices suitable for attaching to or embedding within a small device such as an earplug. It is also contemplated that MEMS and/or piezoelectric materials may be employed in the microphone or speaker elements of the present invention. Although described relative to a user-wearable device, it will be appreciated that various aspects of the present invention may be employed in a larger-scale version of the invention, and accordingly, such scale is not an inherent limitation of the present invention.
Referring next to FIG. 2 , it will be appreciated that various embodiments of the present invention may be employed to attach the system, depending upon the user's needs. As depicted in the figure, musician 16 would have one or more components of the present invention inserted into her ear(s), and would wear a necklace or strap 70 upon which the control and processing module ( 30 . 40 ) would be attached as a pendant. Alternatively, recognizing that in many cases a necklace/pendant combination would not be desirable, the embodiment of FIG. 3 depicts a belt or strap 80 , to which the module is attached to permit the user to wear the system around his/her arm, waist, etc. It will also be appreciated that various components of the described system may be incorporated into or on clothing or other garments (e.g., pockets, vests, caps, hats, headbands, etc.) and the like to permit ease of use.
Although the present disclosure is directed to embodiments wherein the acoustical signal processing apparatus is used by musicians, it is also contemplated that such devices may be used in other applications, particularly those where the user wants or needs to have control over not only the direction of sound that is partially attenuated, but possibly over the frequency range of sound as well. For example, the present invention is contemplated for use by persons having learning disabilities, where the person is highly sensitive to sound or certain frequencies, whereby the person could employ the present invention to reduce background noise to permit the person to study or perform in an uninterrupted fashion. The present is also believed to be suitable for use by musical instructors and those working in or exposed to similar environments.
As illustrated in FIG. 1 , each of the microphones or similar acoustic sensing means 24 A, B generate an output signal that is transferred or transmitted via wires or traces on a circuit board to a signal processing device 30 . As noted previously, one or more of the microphones in the acoustic sensing means or microphone may provide its output to an amplifier or pre-amplifier ( 26 ) to amplify the signal produced by the microphone, preferably on-board, before the signal is transferred or transmitted via the wires or traces. Use of the amplifier is believed to improve the signal-to-noise ratio of the input signal that transferred to the signal processing device 30 .
The signal processing device 30 produces, in response to the input audio signals, at least one output signal. The present invention contemplates, as described in detail below, that the signal processor may operate in response to user selections, adjustments or preferences, whereby the output signals will be adjusted in accordance with the user's preferences. In one embodiment, the signal processing device provides a uniform frequency response such that the output acoustic signal spectrum level is generally reflective of an input acoustic signal spectrum level, while maintaining the directionality thereof.
The signal processing device may comprise a simple set of potentiometers and associated thumbwheels 410 as used in conventional hearing aids to adjust volume levels, a suitable amplifier such as an operational amplifier capable of driving the output speakers, headset, or “ear buds” along with suitable resistors and capacitors commonly used in conventional circuit designs well known to those skilled in the art of analog circuit design. In the embodiments depicted, the additional feature of a mixing circuit 42 has been added. Further details of an exemplary design will be described below relative to FIG. 4 . The device would preferably be responsive to acoustic signals over a full frequency range of about 20 Hz to 20 KHz; albeit a reduced range of 50 Hz to 15 KHz may prove to be acceptable.
In one embodiment, various component manufacturers may supply suitable microphone and/or ear buds; for example, Knowles Acoustics, a division of Knowles Electronics LLC has an extensive selection of such microphones (SiSonic, e.g., Model SP0103) and speaker elements. Presently electret microphones from Knowles are being used. As shown, for example, in FIGS. 4N and 6 microphones 24 look like very small cylinders—with a hole 25 at one end where the sound enters and three pads (not shown) on the back that mount on the PWBA 23 . in the Embodiment depicted in FIG. 4N the microphone assembly looks a bit like a PWBA with back to back microphones in it's center. The microphones may also be in purchased from Radio Shack® and are basically the same type of electret device, but appear to be more sensitive, and may include some sort of circuit inside them.
It is also important to note that the signal processing device should be suitable for interfacing with MEMS-type devices, including microphones and/or speakers, and should include driver circuitry suitable for conditioning the signals to/from such devices. Although MEMS-type microphones are understood to be considered “noisy” they may nonetheless prove to be suitable, and even desirable from a size reduction perspective, for some embodiments of the present invention.
Referring again to FIG. 1 , an acoustic output port 50 is provided in the form of an array of speakers 54 A, B, or similar elements, for releasing the output acoustic signal produced by the signal processing device 30 . In particular, the present invention contemplates the use of a prototype earphone using CMOS-MEMS micromachining techniques that is audible from 1 to 15 kHz and was produced by John J. Neumann, Jr. and Kaigham J. Gabriel at Carnegie Mellon University. Other possibilities are miniature speaker assemblies such as the Knowles balanced armature speakers and “ear buds” such as the Shure E-series earphones. In one embodiment, the invention includes a conformable and/or molded portion that is inserted into the user's ear canal. The conformable or molded material occludes the canal of the user's ear and thereby substantially prevents the user's perception of the ambient sound except what has been processed and output by the speaker or output port 50 .
Having described the basic configuration of an embodiment of the present invention, attention is now turned to additional features that may be incorporated with or in the “smart ear plug” device. More specifically, the user controls 40 are intended to provide adjustment capability for the present invention. In one embodiment, the signal processing device processes the input signals in a manner so that the output acoustic signal includes an attenuated signal from at least one of the plurality of directions. To control the level of attenuation and the direction, it is contemplated that a balance/fader or similar signal direction adjustment, for example mixing circuit 42 , may be employed in combination with a volume or attenuation control. For example, such a device may be employed to reduce the volume (higher signal attenuation) of the trombones located behind the user (e.g., position B), while not reducing the volume of signals from the balance of the orchestra in front of the user (e.g., position A).
As noted above, the output port 50 preferably comprises an array of miniature (e.g., MEMS) speakers, each of which receives and is responsive to one of a plurality of output signals generated by the signal processor 30 .
Although depicted in a larger configuration in FIG. 1 , for purposes of illustration, the present invention ideally fits near or inside the ear channel 19 in much the same way current “ear plugs” do. Alternately, the maximum size would be that of current miniature commercial hearing aids or similar devices, and may include one or more directional microphones that are spaced apart from the processor and speaker array. One goal is that such devices remain essentially invisible to an audience, as well as have minimal effect on the comfort of the user.
Having generally described an embodiment of the invention, and various applications thereof, attention is now turned to FIGS. 3 and 4 where an embodiment of the invention is depicted as a prototype. Here again, the figures (e.g., FIG. 4J ) depict an acoustic signal processing system for processing acoustic signals in accordance with a user's preference. On each ear bud 50 L and 50 R (L and R indicating left and right sides, respectively), there is positioned at least one microphone array 24 A, 24 B. As will be appreciated, each microphone or microphone array generates a plurality of input signals in response to acoustic vibrations received by the microphone. Each input signal represents an acoustic input from one of a plurality of directions relative to said microphone array.
More specifically, the array 24 A-B in a user's left ear would sense acoustic energy (e.g., sound, noise) perceptible from the user's left side. In one embodiment, microphone 24 A-B is a micro-electromechanical system as described above. Moreover, the ear bud 50 L may be oriented so that the microphones 24 A and 24 B are, respectively, oriented toward the front and rear. The system depicted in FIG. 3 further includes, on the ear bud, a baffle 130 (which may be circuit board 23 ), wherein the baffle separates the first ( 24 A) and second ( 24 B) microphones of the microphone array. It will be further appreciated that the baffle may be made from a resilient material, albeit one that preferably does not transfer acoustic energy to better assure the independent operation of each of the microphones. Moreover, the baffle 130 may be integrated with, or comprise, the circuit board 23 upon which the components described below may be mounted, possibly including connections made through RTV silicone rubber or similar edge connection means. Although not specifically depicted, one or more surfaces 132 of the baffle 130 may be parabolic in shape or have a configuration that selectively focuses or directs the acoustic energy from at least one direction toward the microphone. It should also be appreciated that although shown with two microphones on each of the left and right sides, the present invention is not to be so limited, and may include three or more microphones in an array on each ear bud in order to improve the user's sense of directionality.
Also contained in each ear bud ( 50 L, 50 R) is amplification and drive circuitry associated with the microphones (see FIG. 5 ), as well as at least one speaker 54 . The speaker 54 operates to generate an output acoustic signal in response to an output signal from the signal processing device 30 , which will now be described in more detail relative to FIG. 5 . As depicted in FIG. 5 , the signal processing circuitry 30 and user controls 40 operate to process the inputs of microphones 24 A, 24 B (left and right) to produce output signals for speakers 54 . Although depicted as an embodiment employing operational amplifiers (LT1678), it will be appreciated that the operational amplifier or alternative devices (e.g., circuit, integrated circuit, transistor) must be suitable to both amplify the signal from the microphone (e.g., electret) and to drive the headset. It may be possible to employ standard and known components in this regard or to design alternative circuits. The schematic diagram of FIG. 5 illustrates parallel channels (left and right), both of which utilize an operational amplifier (op-amp) 150 . Power is supplied from a plurality of batteries 170 ( FIG. 3 , or the battery pack 170 depicted in FIGS. 4J and 4L ), and input voltage protection is provided by a diode 174 , which not only serves the purpose of a protection diode, but also provides an indication of the operation of the system.
Batteries 170 may be of various types, and the present invention further contemplates the use of a rechargeable battery array, where the power provided to operate the system is supplied from Nickel based or Lithium-Ion type battery(ies). In such an embodiment, the system includes one or more commercially available components such as integrated circuits that may be incorporated to facilitate the continued use of the system without having to replace batteries (e.g., battery charging components available from Linear Technology, Inc. of Milpitas, Calif.). It will be appreciated that such devices, and applications thereof, are commonly known for cellular telephones, personal digital assistants, laptop computers and various other electronic devices and games. The power-on state of user controls 40 , may be indicated by a LED or similar light indicator 415 , and additional indicators or display components may be present to provide feedback to the user.
Exemplary values for the various components are indicated directly on the schematic.
As depicted, for example in FIGS. 4A-4B , the system includes a separate housing for the signal processing device 30 , and is connected to the ear buds 50 L and 5 OR by a wire harness 51 having a removable, multi-pin connector 53 on the end thereof. Connector 53 is attached to signal processing device 30 via the connector depicted in FIG. 4D . The device 30 and an associated battery pack 170 may be attached to the user using a belt (belt clip shown) or attached to the user's clothing or stored in a pocket, depending upon the user's preference.
The signal processing device 30 operates to produce, in response to the plurality of input signals from the microphones 24 , at least one output signal, preferably one output signal on each channel (left and right)—although a single-channel system may be employed for cost reasons in limited-capability applications. The signal processing device is characterized by a uniform frequency response such that an output acoustic signal spectrum level is generally reflective of an input acoustic signal spectrum level. It is a further characteristic of the signal processing device 30 that it operates to generate the output acoustic signal with at least a partially attenuated signal from at least one of the plurality of directions (e.g. front, rear). It will be appreciated that the characteristics of the components used may further be used to select or control the amount of attenuation achieved by the system. More importantly, the dual potentiometers (linked for front and rear in the circuit of FIG. 4 ) cause the signal processing device 30 to operate to attenuate the output signal in response to a user adjustable control. As will be appreciated, the output signals (left and right) are directed to a plurality of speakers 54 L and 54 R, where each of the speakers receive and are responsive to one of the plurality of output signals.
In yet another embodiment it is contemplated that one or more of the hard wire connections between one or more of the microphones, signal processing unit and/or earphones (speakers) may employ short distance, low power, wireless technology including, but not limited to, radio frequency (approx. 100 MHz), Bluetooth™ (2.4 GHz), or infrared (100 GHz) or similar wireless transceivers. One impetus for such a feature, as well as cost, is found in the ability to mitigate the entanglement of wires with the musicians and their instruments. An additional advantage of wireless connectivity may be the use of microphones that are more remotely situated (spaced apart from the user) in certain circumstances where it may be desirable to do so. Furthermore, and for example, the audio control unit could be placed on a music stand, for instance, so as it was out of view of the audience but easily accessible by the musician without the added annoyance of being wired to the stand.
In accordance with another aspect of the invention a headset, similar to traditional headphones used in recording studios, is provided whereby the ear cups isolate the ears from all ambient sound. Strategically positioned microphones, within the headband, attached to the ear cups, etc., provide an acoustically blended output to the earphone as produced by the audio control unit, also located within the headset. The signal processing unit samples and modifies each of a plurality of microphone inputs as a function of the musicians adjustment of the gain controls The advantage of this self contained unit is based on convenience and comfort and is may be most practical while rehearsing—in order to preserve audio acuity as well as selective attenuation of the more profound instruments. An additional advantage to the headset version, as well as the ear bud, is to allow the microphones to “follow” the sound as you turn your head thereby providing a more realistic acoustic rendition. For example the microphones might possibly be located in eyeglasses, a hat, a hairpiece or similar apparel worn on the user's head.
In recapitulation, the present invention is a method and apparatus for controlling a user's auditory input using a smart earplug. It is, therefore, apparent that there has been provided, in accordance with the present invention, a method and apparatus for acoustic control. While this invention has been described in conjunction with preferred embodiments thereof, it is evident that many alternatives, modifications, and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims.
The claims, as originally presented and as they may be amended, encompass variations, alternatives, modifications, improvements, equivalents, and substantial equivalents of the embodiments and teachings disclosed herein, including those that are presently unforeseen or unappreciated, and that, for example, may arise from applicants/patentees and others.