Image Sharpness by Mark Boys

UNITED STATES PATENT AND TRADEMARK OFFICE Ul\TfE]) STI\TES ])EPA RTME'IT OF COMMERCE United States Patent and Trademark Office Adill",. COMMISSIO'JER FOR PATENTS PO Box 1450 Alexandria, Virgmia 22313-1450 \VVi\V.uspto.gOY

APPLICATION NUMBER

FILING or

371 (c) DATE

FIL FEE REC'D

ATTY.DOCKET.NO

12/908,161

10/20/2010

682

P882

24739 CENTRAL COAST PATENT AGENCY, INC 3 HANGAR WAY SUITE D WATSONVILLE, CA 95076

18 5 CONFIRMATION NO. 3367

FILING RECEIPT

111111111111111111111111]~!I]~~I~~I~~11~~~~Jj ~llllllllllllllllllllllllll Date Mai led: 11/04/201 0

Receipt is acknowledged of this non-provisional patent application. The application will be taken up for examination in due course. Applicant will be notified as to the results of the examination. Any correspondence concerning the application must include the following identification information: the U.S. APPLICATION NUMBER, FILING DATE, NAME OF APPLICANT, and TITLE OF INVENTION. Fees transmitted by check or draft are subject to collection. Please verify the accuracy of the data presented on this receipt. If an error is noted on this Filing Receipt, please submit a written request for a Filing Receipt Correction. Please provide a copy of this Filing Receipt with the changes noted thereon. If you received a "Notice to File Missing Parts" for this application, please submit any corrections to this Filing Receipt with your reply to the Notice. When the USPTO processes the reply to the Notice, the USPTO will generate another Filing Receipt incorporating the requested corrections Appl icant( s) Rodney Shaw, Aptos, CA; Power of Attorney: The patent practitioners associated with Customer Number 24739 Domestic Priority data as claimed by applicant Foreign Applications

Permission to Access - A proper Authorization to Permit Access to Application by Participating Offices (PTO/SB/39 or its equivalent) has been received by the USPTO. If Required, Foreign Filing License Granted: 11/01/2010 The country code and number of your priority application, to be used for filing abroad under the Paris Convention, is US 12/908,161 Projected Publication Date: 04/26/2012 Non-Publication Request: No Early Publication Request: No ** SMALL ENTITY **

page 1 of 3

Title

Sharpness in Digital Images Preliminary Class

382

PROTECTING YOUR INVENTION OUTSIDE THE UNITED STATES Since the rights granted by a U.S. patent extend only throughout the territory of the United States and have no effect in a foreign country, an inventor who wishes patent protection in another country must apply for a patent in a specific country or in regional patent offices. Applicants may wish to consider the filing of an international application under the Patent Cooperation Treaty (PCT). An international (PCT) application generally has the same effect as a regular national patent application in each PCT-member country. The PCT process simplifies the filing of patent applications on the same invention in member countries, but does not result in a grant of "an international patent" and does not eliminate the need of applicants to file additional documents and fees in countries where patent protection is desired. Almost every country has its own patent law, and a person desiring a patent in a particular country must make an application for patent in that country in accordance with its particular laws. Since the laws of many countries differ in various respects from the patent law of the United States, applicants are advised to seek guidance from specific foreign countries to ensure that patent rights are not lost prematurely. Applicants also are advised that in the case of inventions made in the United States, the Director of the USPTO must issue a license before applicants can apply for a patent in a foreign country. The filing of a U.S. patent application serves as a request for a foreign filing license. The application's filing receipt contains further information and guidance as to the status of applicant's license for foreign filing. Applicants may wish to consult the USPTO booklet, "General Information Concerning Patents" (specifically, the section entitled "Treaties and Foreign Patents") for more information on timeframes and deadlines for filing foreign patent applications. The guide is available either by contacting the USPTO Contact Center at 800-786-9199, or it can be viewed on the USPTO website at http://www.uspto.gov/web/offices/pac/doc/general/index.html. For information on preventing theft of your intellectual property (patents, trademarks and copyrights), you may wish to consult the U.S. Government website, http://www.stopfakes.gov. Part of a Department of Commerce initiative, this website includes self-help "toolkits" giving innovators guidance on how to protect intellectual property in specific countries such as China, Korea and Mexico. For questions regarding patent enforcement issues, applicants may call the U.S. Government hotline at 1-866-999-HALT (1-866-999-4158).

LICENSE FOR FOREIGN FILING UNDER Title 35, United States Code, Section 184 Title 37, Code of Federal Regulations, 5.11 & 5.15 GRANTED

The applicant has been granted a license under 35 U.S.C. 184, if the phrase "IF REQUIRED, FOREIGN FILING LICENSE GRANTED" followed by a date appears on this form. Such licenses are issued in all applications where the conditions for issuance of a license have been met, regardless of whether or not a license may be required as page 2 of 3

set forth in 37 CFR 5.15. The scope and limitations of this license are set forth in 37 CFR 5.15(a) unless an earlier license has been issued under 37 CFR 5.15(b). The license is subject to revocation upon written notification. The date indicated is the effective date of the license, unless an earlier license of similar scope has been granted under 37 CFR 5.13 or 5.14. This license is to be retained by the licensee and may be used at any time on or after the effective date thereof unless it is revoked. This license is automatically transferred to any related applications(s) filed under 37 CFR 1.53(d). This license is not retroactive. The grant of a license does not in any way lessen the responsibility of a licensee for the security of the subject matter as imposed by any Government contract or the provisions of existing laws relating to espionage and the national security or the export of technical data. Licensees should apprise themselves of current regulations especially with respect to certain countries, of other agencies, particularly the Office of Defense Trade Controls, Department of State (with respect to Arms, Munitions and Implements of War (22 CFR 121-128)); the Bureau of Industry and Security, Department of Commerce (15 CFR parts 730-774); the Office of Foreign AssetsControl, Department of Treasury (31 CFR Parts 500+) and the Department of Energy. NOT GRANTED No license under 35 U.s.C. 184 has been granted at this time, if the phrase "IF REQUIRED, FOREIGN FILING LICENSE GRANTED" DOES NOT appear on this form. Applicant may still petition for a license under 37 CFR 5.12, if a license is desired before the expiration of 6 months from the filing date of the application. If 6 months has lapsed from the filing date of this application and the licensee has not received any indication of a secrecy order under 35 U.S.C. 181, the licensee may foreign file the application pursuant to 37 CFR 5.15(b).

page 3 of 3

To: From: Cc: Subject:

anantha@formulateip.com,officeactions@C ENTRALCOASTPATENT. COM, plambuth@centralcoastpatent.< PAIR_eOfficeAction@uspto.gov PAIR_eOfficeAction@uspto.gov Private PAIR Correspondence Notification for Customer Number 24739

Nov 04, 201005:49:26 AM Dear PAIR Customer: CENTRAL COAST PATENT AGENCY, INC 3 HANGAR WAY SUITE D WATSONVILLE, CA 95076 UNITED STATES The following USPTO patent application(s) associated with your Customer Number, 24739 , have new outgoing correspondence. This correspondence is now available for viewing in Private PAIR. The official date of notification of the outgoing correspondence will be indicated on the form PTOL-90 accompanying the correspondence. Disclaimer: The list of documents shown below is provided as a courtesy and is not part of the official file wrapper. The content of the images shown in PAIR is the official record. Application 12908161

Document APP.FILE.REC

Mailroom Date 11/04/2010

Attorney Docket No. P882

To view your correspondence online or update your email addresses, please visit us anytime at https:llsportal.uspto.gov/secure/myportal/privatepair. If you have any questions, please email the Electronic Business Center (EBC) at EBC@uspto.gov with 'e-Office Action' on the subject line or call 1-866-217-9197 during the following hours: Monday - Friday 6:00 a.m. to 12:00 a.m. Thank you for prompt attention to this notice, UNITED STATES PATENT AND TRADEMARK OFFICE PATENT APPLICATION INFORMATION RETRIEVAL SYSTEM

PTO/SB/14 (07-07) Approved for use through 06/30/2010. OMB 0651-0032 U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE

Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it contains a valid OMB control number.

Application Data Sheet 37 CFR 1.76 Title of Invention

Attorney Docket Number

P882

Application Number

Sharpness in Digital Images

The application data sheet is part of the provisional or non provisional application for which it is being submitted. The following form contains the bibliographic data arranged in a format specified by the United States Patent and Trademark Office as outlined in 37 CFR 1.76. This document may be completed electronically and submitted to the Office in electronic format using the Electronic Filing System (EFS) or the document may be printed and included in a paper filed application.

Secrecy Order 37 CFR 5.2

Portions or all of the application associated with this Application Data Sheet may fall under a Secrecy Order pursuant to 37 CFR 5.2 (Paper filers only. Applications that fall under Secrecy Order may not be filed electronically.)

Applicant Information:

I Remove I

Applicant 1

IOParty of Interest under 35 U.S.C. 118

Applicant Authority (!)Inventor I OLegal Representative under 35 U.S.C. 117 Prefix Given Name

Middle Name

Family Name

Rodney

Shaw

Residence Information (Select One) (!) US Residency City

Aptos

State/Province

Citizenship under 37 CFR 1.41(b) i

Non US Residency

Suffix

Active US Military Service

Country of Residence i

Mailing Address of Applicant: 281 Ventana Way

Address 1 Address 2 City

Aptos

Postal Code

95003

State/Province

Countryi

All Inventors Must Be Listed - Additional Inventor Information blocks may be generated within this form by selecting the Add button.

Add

Correspondence Information: Enter either Customer Number or complete the Correspondence Information section below. For further information see 37 CFR 1.33(a).

An Address is being provided for the correspondence Information of this application.

Customer Number

24739

Email Address

info@centralcoastpatenl.com

Add

Email I

IRemove Emaill

Application Information: Title of the Invention

Sharpness in Digital Images

Attorney Docket Number

P882

Application Type

Nonprovisional

Subject Matter

Utility

Suggested Class (if any)

Small Entity Status Claimed

Sub Class (if any)1

Suggested Figure for Publication (if any)

Suggested Technology Center (if any) Total Number of Drawing Sheets (if any) EFS Web 2.2.2

PTO/SB/14 (07-07) Approved for use through 06/30/2010. OMB 0651-0032 U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it contains a valid OMB control number.

Application Data Sheet 37 CFR 1.76 Title of Invention

Attorney Docket Number

P882

Application Number

Sharpness in Digital Images

Publication Information:

D D

Request Early Publication (Fee required at time of Request 37 CFR 1.219)

Request Not to Publish.

I hereby request that the attached application not be published under 35 U.S.C. 122(b) and certify that the invention disclosed in the attached application has not and will not be the subject of an application filed in another country, or under a multilateral international agreement, that requires publication at eighteen months after filing.

Representative Information: Representative information should be provided for all practitioners having a power of attorney in the application. Providing this information in the Application Data Sheet does not constitute a power of attorney in the application (see 37 CFR 1.32). Enter either Customer Number or complete the Representative Name section below. If both sections are completed the Customer Number will be used for the Representative Information during processing. Please Select One:

C!J

Customer Number

24739

Customer Number

10 US Patent Practitioner 10

Limited Recognition (37 CFR 11.9)

Domestic Benefit/National Stage Information: This section allows for the applicant to either claim benefit under 35 U.S.C. 119(e), 120, 121, or 365(c) or indicate National Stage entry from a PCT application. Providing this information in the application data sheet constitutes the specific reference required by 35 U.S.C. 119(e) or 120, and 37 CFR 1.78(a)(2) or CFR 1.78(a)(4), and need not otherwise be made part of the specification.

I Remove I

Prior Application Status Application Number

Continuity Type

Prior Application Number

Filing Date (YYYY-MM-DD)

Additional Domestic Benefit/National Stage Data may be generated within this form by selecting the Add button.

Add

Foreign Priority Information: This section allows for the applicant to claim benefit of foreign priority and to identify any prior foreign application for which priority is not claimed. Providing this information in the application data sheet constitutes the claim for priority as required by 35 U.S.C. 119(b) and 37 CFR 1.55(a).

I Remove I Application Number

Country i

Parent Filing Date (YYYY-MM-DD)

Priority Claimed

0 Additional Foreign Priority Data may be generated within this form by selecting the Add button.

Add

Yes

C!J

Assignee Information: Providing this information in the application data sheet does not substitute for compliance with any requirement of part 3 of Title 37 of the CFR to have an assignment recorded in the Office. Assignee 1

EFS Web 2.2.2

I Remove I

Application Data Sheet 37 CFR 1.76 Title of Invention

Attorney Docket Number

P882

Application Number

Sharpness in Digital Images

If the Assignee is an Organization check here. Prefix

Given Name

Middle Name

Family Name

Suffix

Mailing Address Information: Address 1 Address 2 City

State/Province

Country il

Postal Code

Phone Number

Fax Number

Email Address Additional Assignee Data may be generated within this form by selecting the Add button.

Add

Signature: A signature of the applicant or representative is required in accordance with 37 CFR 1.33 and 10.18. Please see 37 CFR 1.4(d) for the form of the signature. Signature First Name

!Donald R. Boys! Donald

Last Name

Boys

Date (YYYY-MM-DD)

2010-10-20

Registration Number

35074

This collection of information is required by 37 CFR 1.76. The information is required to obtain or retain a benefit by the public which is to file (and by the USPTO to process) an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.14. This collection is estimated to take 23 minutes to complete, including gathering, preparing, and submitting the completed application data sheet form to the USPTO. Time will vary depending upon the individual case. Any comments on the amount of time you require to complete this form and!or suggestions for reducing this burden, should be sent to the Chief Information Officer, U.S. Patent and Trademark Office, U.S. Department of Commerce, P.O. Box 1450, Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. SEND TO: Commissioner for Patents, P.O. Box 1450, Alexandria, VA 22313路1450.

EFS Web 2.2.2

Privacy Act Statement

The Privacy Act of 1974 (P .L. 93-579) requires that you be given certain information in connection with your submission of the attached form related to a patent application or patent. Accordingly, pursuant to the requirements of the Act, please be advised that: (1) the general authority for the collection of this information is 35 U.S.C. 2(b)(2); (2) furnishing of the information solicited is voluntary; and (3) the principal purpose for which the information is used by the U.S. Patent and Trademark Office is to process and/or examine your submission related to a patent application or patent. If you do not furnish the requested information, the U.S. Patent and Trademark Office may not be able to process and/or examine your submission, which may result in termination of proceedings or abandonment of the application or expiration of the patent. The information provided by you in this form will be subject to the following routine uses: 1.

The information on this form will be treated confidentially to the extent allowed under the Freedom of Information Act (5 U.S.C. 552) an the Privacy Act (5 U.S.C. 552a). Records from this system of records may be disclosed to the Department of Justice to determine whether the Freedom of Information Act requires disclosure of these records.

A record from this system of records may be disclosed, as a routine use, in the course of presenting evidence to a court, magistrate, or administrative tribunal, including disclosures to opposing counsel in the course of settlement negotiations.

A record in this system of records may be disclosed, as a routine use, to a Member of Congress submitting a request involving an individual, to whom the record pertains, when the individual has requested assistance from the Member with respect to the subject matter of the record.

A record in this system of records may be disclosed, as a routine use, to a contractor of the Agency having need for the information in order to perform a contract. Recipients of information shall be required to comply with the requirements of the Privacy Act of 1974, as amended, pursuant to 5 U.S.C. 552a(m).

A record related to an International Application filed under the Patent Cooperation Treaty in this system of records may be disclosed, a a routine use, to the International Bureau of the World Intellectual Property Organization, pursuant to the Patent Cooperation Treaty.

A record in this system of records may be disclosed, as a routine use, to another federal agency for purposes of National Security review (35 U.S.C. 181) and for review pursuant to the Atomic Energy Act (42 U.S.C. 218(cÂť.

A record from this system of records may be disclosed, as a routine use, to the Administrator, General Services, or his/her designee, during an inspection of records conducted by GSA as part of that agency's responsibility to recommend improvements in records management practices and programs, under authority of 44 U.S.C. 2904 and 2906. Such disclosure shall be made in accordance with the GSA regulations governing inspection of records for this purpose, and any other relevant (i.e., GSA or Commerce) directive. Such disclosure shall not be used to make determinations about individuals.

A record from this system of records may be disclosed, as a routine use, to the public after either publication of the application pursuan to 35 U.S.C. 122(b) or issuance of a patent pursuant to 35 U.S.C. 151. Further, a record may be disclosed, subject to the limitations of 37 CFR 1.14, as a routine use, to the public if the record was filed in an application which became abandoned or in which the proceedings were terminated and which application is referenced by either a published application, an application open to public inspections or an issued patent.

A record from this system of records may be disclosed, as a routine use, to a Federal, State, or local law enforcement agency, if the USPTO becomes aware of a violation or potential violation of law or regulation.

EFS Web 2.2.2

DECLARATION AND POWER OF ATTORNEY FOR PATENT APPLICATION ATTORNEY DOCKET NO.P882 As a below named inventor, I hereby declare that: My residence, post office address and citizenship are as stated below next to my name. I believe I am the original, first and sole inventor (if only one name is listed below) or an original, first and joint inventor (if plural names are listed below) of the subject matter which is claimed and for which a patent is sought on the invention entitled: Sharpness in Digital Images the specification of which (check one) ~ is attached hereto. was filed on: _ _ Application Serial No. _ _ and was amended on _ _ (If applicable) I hereby state that I have reviewed and understand the contents of the above-identified specification, including the claims, as amended by any amendment referred to above. I acknowledge the duty to disclose information which is material to the examination ofthis application in accordance with Title 37, Code of Federal Regulations, s 1.56 (a). In the case that the present application is a continuation-in-part application, I further acknowledge the duty to disclose material information as defmed in 37 CFR s 1.56(a) which became available between the filing date of tile prior application and the filing date of the present application. I hereby claim foreign priority benefits under Title 35, United States Code sll9 of any foreign applications for patent or inventor's certificate listed below and have also identified below any foreign application for patent or inventor's certificate having a filing date before that ofthe application on which priority is claimed: Prior Foreign Application(s) (Country) (Number) (Day/MonthIYear Filed)

o o o

(Number)

(Country)

(Day/Month/Year Filed)

I hereby claim the benefit under Title 35, United States Codes, 120 and 119 of any United States application(s) listed

below and, insofar as the subject matter of each of the claims of this application is not disclosed in the prior United States application in the manner provided by the first paragraph of Title 35, United States Code, 8112, I acknowledge the duty to disclose material information as dermed in Title 37, Code of Federal Regulations, 8156(a) which occurred betwecn the filing date of the prior application and the national or peT international filing date of this application. (Application Serial No.): _ _ (Application Serial No.); _ _ (Application Serial No.): _ _ (Application Serial No.): _ _ (Application Serial No.): _ _

(Filing Date): (Filing Date): (Filing Date): (Filing Date): (Filing Date):

__ __ __ __ __

(Statns): _ __ (Statns): _ __ (Statns): _ __ (Status): _ __ (Status): _ __

POWER OF ATTORNEY: As a named inventor, I hereby appoint:

[;gI

Practitioners at customer number:

24739

D Practitioners: Name:

Registration number

to prosecute this application and transact all business in the Patent and Trademark Office connected therewith. Please send all correspondence practitioners at:

~ The practitioners at the customer number indicated abovc

D Customer number: 24739

Page 2 DECLARATION AND POWER OF ATTORNEY FOR PATENT APPLICATION ATTORNEY DOCKET NO. PSS2 Authorization To Permit Access To Application by Participating Offices The above-signed hereby grants the USPTO authority to provide the European Patent Office (EPO), the Japan Patent Office (JPO), the Korean Intellectual Property Office (KIPO), the World Intellectual Property Office (WIPO), and any other intellectual property offices in which a foreign application claiming priority to the above-identified patent application is filed access to the above-identified patent application. See 37 CFR 1.14(c) and (h). In accordance with 37 CFR 1.14(h)(3), access will be provided to a copy of the above-identified patent application with respect to: 1) the above-identified patent application-as-filed; 2) any foreign application to which the above-identified patent app1ication claims priority under 35 U.S.C. 119(a)-(d) if a copy of the foreign application that satisfies the certified copy requirement of 37 CFR 1.55 has been filed in the above-identified patent application; and 3) any U.S. application-as-filed from which benefit is sought in the above-identified patent application.

In accordance with 37 CFR 1.14(c), access may be provided to information concerning the date of filing the Authorization to Permit Access to Application by Participating Offices.

I hereby declare that all statements made herein of my own knowledge arc true and that all statements made on information and belief are believed to be tme; and further that these statements were made with the knowledge that willful false statements and the like so made are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that such willful false statements may jeopardize the validity ofthe application or any patent issued thereon.

I st inventor's signature: _ _ _ _-짜-_--'-'wt---"LO----=------,j'---L._ _-=--"""O::"_ _ _ _ _ _ Dated: Residence: 281 Ventana Way Aptos CA 95003

Post Office Address: Same

Declaration and Power of Attorney- Page 2

Oct J'I '"' 2. 0 10

~ ~

,........;

.....Q ~

.~ ~

\~ ~ I

~ I

\". \.

...D

~ I

Cl)

'e ~~

,..t:j

~ I

'--

c!j

~ I

.~ ~

@ !

(

~)速

\ \

('.l

0 ,.........

,......... 0 ,.........

\ \)

/1 /

('.l

0 ,.........

,.........~.

@ \

o,.........

)

\ \

,.........

o,.........

501

507 2048

2048

Original Image

Final Image

1536

~ Downsizing

502

Algorithm //-- 503

504

505

1536

Upsizing Algorithm

Fi~D ~

Downsized 320 Original ----Âť Iteration I

Second Iteration

"n"th Third ----Âť Iteration >Iteration Chosen 506

480

Fig. 5

(601

602

Select an image to be enhanced for Sharpness

Downsize the image I--->'~I to a substantially smaller resolution producing image "0"

603 Apply a minimal- computation convolution filter to image "0" I to produce an image "n"

(606

"" (604

(605

Add the quotient for each pixel from step 605 to associated pixel ~ value in image "0", to produce an image "I".

607~

>'1

Determine algebraic Divide the algebraic difference for each pixell~(-- difference between each pixel value of "n" in step 604 by integer "n" with each pixel of "0"

(608

Repeat process in step 606, incrementing the image no. by 1 for each iteration, until the image no. is n-l.

Present images "0" through "n" --~>,t>l1 in sequential order for a user to select "best" image for sharpness

,."" ~

( Fig. 6

610

Upsize "best" image back to original larger resolution.

(701

(702

Select a plurality of digital images of similar sharpness quality

Choose one of the images as image "0" to be enhanced for Sharpness

(703 ~

Apply a minimal- computation convolution filter to "0" to produce an image "n"

706~

,......

("705

Add the quotient for each pixel from step 605 to associated pixel value in image "0", to produce an image "1".

Divide the algebraic difference for each pixel 14 in step 604 by integer "n"

1(--

("707

\It

Repeat process in step 606, incrementing the image no. by 1 for each iteration, until the image no. is n-l.

'W'

704

Determine algebraic difference between each pixel value of "n" with same pixel of "0"

(708 Present images "0" through "n" ---l)1Ii'I1 in sequential order for a user to select "best" image for sharpness

,.'" (709

'W'

( 710

Process each remaining image of I the plurality to produce just the .(-enhanced image of the same number as in step 708

Fig. 7

Note the number of the image the user selects as best image

M 1

1 .9

.7 .6

Local Pixel Value Average

Fig. 8

255

901

(902

Place filter and determine new pixel value at first pixel position.

Determine algebraic diffence between original and new pixel value

906\

(903

,""

Select multiplier between oand I from graph using segment preference

(905

Save the result of step 905 as new value for image "n"

\It

\It ,904

Add result of step 904 Multiply difference back to original pixel 14:1(-from step 902 by value multiplier from step 903

("907

Place filter at next position and repeat the process until all positions are determined.

--,

Fig. 9

SHARPNESS IN DIGITAL IMAGES

CROSS-REFERENCE TO RELATED APPLICATIONS N/A 5

BACKGROUND OF THE INVENTION

1. Field of the Invention The present invention is in the field of apparatus and techniques for enhancing the 10

apparent visual quality of images stored and presented digitally.

2. Description of Related Art Many techniques exist for processing and filtering images displayed in a computerized system as two-dimensional matrices of pixels, typically presented in a 15

rectangular matrix. A digital display presents pixels in color and brightness according to values stored in memory for each pixel. For color, for example, there will be a separate value in the RGB system for red (R), green (G) and blue (B). In an eight-bit computer process, the values for each pixel for each attribute range from 0 to 255, which is 28. For any of a wide variety of reasons, digital images may be less than optimal in image quality

as viewed by a person, and many commercial programs and techniques exist and are available for altering the pixel values to improve the apparent quality of a digital image. To improve the apparent quality of an image, an original image may be altered in one or more of several attributes, such as brightness, contrast, color or what is known in the art as sharpness. The present patent application is in the field of altering sharpness of digital

images to improve apparent quality to an observer. To change apparent sharpness in a pixilated image requires changing individual pixel values in relation to the values of surrounding (proximal) pixels. One technique well known in the art for changing apparent sharpness is controlled application of what are known in the art as convolution filters. References in the art to convolution filters

-2and their uses are numerous. For example RoboRealm at http://www.roborealm.com/help/Convolution.php has a good description of convolution filters and their uses. Very generally, the way a convolution filter works is that a group of typically 5

adjacent pixels in the image to be enhanced is considered, the group having a central pixel whose value is to be modified, dependent in some fashion on the value of the adjacent pixels. The value of each pixel in the group is multiplied by a pre-determined number, which theoretically may be different for each pixel in the group, the results are added, the sum is divided by the number of pixels in the group (average), the determined

average value is divided by a number that is a function of the multipliers for each pixel, and the final value is applied as a new value for the central pixel. The values for the other pixels in the group are not changed. Next the filter is repositioned to have a different central pixel, and asserted again just as above, to determine a new value for the new center pixel, which may be a pixel adjacent to the pixel just previously altered. In

this way new values are determined for almost all pixels in the image Edge pixels may be unchanged because of the geometry of the filter, and may be treated separately, such as leaving with the original value, which in practice has little if any noticeable effect on the enhanced image. In the art of image enhancement, an important consideration is presenting

relatively small changes in an image to a person for determination of improvement, because the optimum sharpness is a matter of opinion and viewing conditions. If changes in an image presented to a person are quite large, it is difficult and time consuming to select a preferred image. It is a good idea, therefore, to be able to present enhanced images such that a newly enhanced image differs from an original or a previous

enhancement in what the present inventor chooses to call a "just appreciable visual sharpness difference" (JAVSD). An historic problem with convolution filters for enhancement of apparent sharpness, is that the process is computation intensive, requiring in many cases substantial computer power. Small filters are possible (minimum number of pixels

-3typically nine), and can be defined so the divisor for the last step is one, but the effect of processing an image with such a minimal-computation filter is typically a very large and unacceptable change in sharpness, far beyond what one might consider a JAVSD. To get a small appreciable variation in sharpness, typically a JAVSD, larger filters with much 5

more computational power required have to be used. This is impractical for very large images (many megapixels), or for cameras, iPods, cell phones, and other devices limited in computational power. What is critically needed in the art of image enhancement is a solution in which minimal-computation filters may be used, and at the same time enhanced images may

still be presented to a user in just-appreciable visual differences. Also needed is a way to process very large, high-pixel density (high resolution) with a minimum of computational power, therefore in essentially real time. The present invention provides this muchneeded solution.

15 BRIEF SUMMARY OF THE INVENTION

The inventor in the present case has considerable experience in image enhancement technology, and has been less than satisfied with the time and computing 20

power necessary to enhance images visually, especially in the attribute of sharpness, as known in the art. As a consequence, the inventor has developed a unique system and process that accomplishes the desired end with a minimum in time and computing power.

In this invention, in one embodiment, a method for enhancing sharpness for a digital image is provided, comprising the steps of ( a) in a display of a computerized appliance, 25

selecting an image to be enhanced in sharpness; (b) downsizing the selected image by a standard downsizing algorithm executing on the computerized appliance to produce an image 0 at resolution substantially less than resolution of the original image selected in step (a); (c) applying a convolution filter to image 0 to produce an image n with enhanced sharpness, where n is an integer; (d) subtracting pixel values for pixels of image n from

-4corresponding pixels for image 0, saving the differences; (e) dividing the differences in step (d) by integer n, and saving the quotients; (f) adding the quotients from step (e) to values for corresponding pixels in image 0 to produce an image 1, then to values of pixels for image 1 to produce an image 2, and repeating until an image n-l is produced; (g) 5

presenting images 0 through n to a user for selection of a best image for sharpness; and (h) upsizing the user-selected image by a standard upsizing algorithm back to the resolution of the image selected in step (a). In one embodiment the convolution filter is a 3 x 3 filter with multipliers of -1 at all cells but the center cell. Also in one embodiment multiplier at the center cell is 9,

producing a divisor of 1 for application of the filter. In some embodiments n = 10 or greater. In another aspect of the invention a system for enhancing sharpness for a digital image is provided, comprising a computerized appliance having a digital display and executing software from a machine-readable medium, the software providing a

mechanism enabling a user to select an image to be enhanced, a downsizing algorithm enabling the user to downsize the selected image to a resolution substantially less than the than resolution of the original image selected, a convolution filter and functions for applying the convolution filter to stored images to produce images enhanced for sharpness, and an up sizing algorithm enabling the user to upsize an image to a higher

resolution. The user selects an image to be enhanced in sharpness, the image is downsized to produce an image 0 at resolution substantially less than resolution of the original image selected, the convolution filter is applied to image 0 to produce an image n with enhanced sharpness, where n is an integer, the pixel values for pixels of image n are subtracted from corresponding pixels for image 0, saving the differences, the differences

are divided by n, saving the quotients, the quotients are added back to the pixel values for image 0 to produce an image 1, and the process is repeated adding the quotients to pixel values of image 1 to produce an image 2, and so forth, until an image n-l is produced, then images 0 through n are presented to the user for selection of a best image for

-5-

sharpness, then the selected image is upsized back to the resolution of the original image selected to be enhanced in sharpness. In one embodiment of the system the convolution filter is a 3 x 3 filter with multipliers of -1 at all cells but the center cell. Also in one embodiment the multiplier at 5

the center cell is 9, producing a divisor of 1 for application of the filter. In some embodiments n = 10 or greater. In another aspect of the invention a method for enhancing sharpness for a digital image is provided comprising the steps of (a) in a display of a computerized appliance, selecting an image to be enhanced in sharpness; (b) downsizing the selected image by a

downsizing algorithm executing on the computerized appliance to produce an image 0 at resolution substantially less than resolution of the original image selected in step (a); (c) applying a convolution filter to image 0 to produce an image n with enhanced sharpness, where n is an integer; (d) subtracting pixel values for pixels of image n from corresponding pixels for image 0, saving the differences; (e) dividing the differences in

step (d) by integer n, and saving the quotients; (f) adding the quotients from step (e) to values for corresponding pixels in image 0 to produce an image 1; (g) displaying image 1 to a user and asking for approval; (h) in case of no approval at step (g), adding the quotients from step (e) to the pixel values for image 1 to produce an image 2; (i) repeating building new images by process of steps (g) and (h) until the user selects one as

best image; and (j) up sizing the user-selected image by an upsizing algorithm back to the resolution of the image selected in step (a). In one embodiment of this method the convolution filter is a 3 x 3 filter with multipliers of -1 at all cells but the center cell. Also in one embodiment the multiplier at the center cell is 9, producing a divisor of 1 for application of the filter. In some cases n

= 10 or greater. In yet another aspect of the invention a system for enhancing sharpness for a digital image is provided, comprising a computerized appliance executing software from a machine-readable medium, the software providing a mechanism enabling a user to select an image to be enhanced, a downsizing algorithm enabling the user to downsize

-6the selected image to a resolution substantially less than the than resolution of the original image selected, a convolution filter and controls for applying the convolution filter to stored images to produce images enhanced for sharpness, and an up sizing algorithm enabling the user to upsize an image to a higher resolution. 5

The user selects an image to be enhanced in sharpness, the image is downsized to

°at resolution substantially less than resolution of the original image selected, the convolution filter is applied to image °to produce an image n with enhanced

produce an image

sharpness, where n is an integer, the pixel values for pixels of image n are subtracted from corresponding pixels for image 0, saving the differences, the differences are divided 10

by n, saving the quotients, the quotients are added back to the pixel values for image

°to

produce an image 1, image 1 is displayed to the user to approve or not as a best image for sharpness, in the case of no approval the saved quotients are added to the pixel values of image 1 to produce an image 2, which is displayed to the user for approval, and the process is repeated until the user selects an image as the best image for sharpness, then 15

the selected image is upsized back to the resolution of the original image selected to be enhanced in sharpness. In one embodiment the convolution filter is a 3 x 3 filter with multipliers of -1 at all cells but the center cell. Also in one embodiment the multiplier at the center cell is 9, producing a divisor of 1 for application of the filter. In some embodiment n = 10 or

greater. In still another aspect of the invention a method for producing a sequence of images enhanced for sharpness is provided, comprising the steps of (a) selecting an image to be enhanced as image 0, (b) applying a convolution filter to image

°to produce an

image n with enhanced sharpness, where n is an integer, (c) subtracting pixel values for 25

pixels of image n from corresponding pixels for image 0, saving the differences, (d) dividing the differences in step (c) by integer n, and saving the quotients, and (e) adding the quotients from step (d) to values for corresponding pixels in image

°to produce an

image 1, then to values of pixels for image 1 to produce an image 2, and repeating until an image n-l is produced.

-7In one embodiment there is a further step for presenting the images as a sequence

of images to a user for selection of one of the images as a best image for sharpness.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

Fig. 1 Fig. 1 is a representation of geometry and placement of a 3 x 3 convolution filter 102 in the art. Fig. 2 is an enlarged view of the filter of Fig. 1, with associated values, to 10

illustrate the computational procedure for asserting the filter at one position. Fig. 3 is a representation of how the filter of Fig. 1 might be moved over an image to produce heightened sharpness for the entire image. Fig. 4 Fig. 4 illustrates a computerized appliance having Internet connection via a wireless network that communicates with a station, thence through a gateway to the

Internet backbone, which represents all of the network interconnections in the Internet network. Fig. 5 illustrates a process for selectively enhancing sharpness of relatively large images in a minimum amount of time, using devices of limited computational power. Fig. 6 is a process flow diagram (flow chart) illustrating steps in a process

according to a preferred embodiment of the present invention. Fig. 7 is a flow diagram illustrating a process for batch processing of similar images in an embodiment of the present invention. Fig. 8 is a diagram for use in preferential sharpening in different segments of an image in an embodiment of the present invention.

Fig. 9 is a flow diagram depicting a process for preferential sharpening in segments based on local pixel value averages indicating relative lightness or darkness in the image in the local vicinity.

-8DETAILED DESCRIPTION OF THE INVENTION

Convolution filters are spatial filters. Spatial filtering is the filtering of an image in the spatial domain. That is, the value of each pixel of the image is modified in 5

contextual relationship to values of neighboring pixels. Consider, for example, a digital image of 320 rows and 480 columns, having a total of 1.536 x 10 5 pixels. The smallest grouping of pixels which associates one pixel with all of its nearest neighbors is typically a 3 x 3 matrix of nine pixels, in which a central pixel is seen surrounded by its eight nearest neighbors.

Fig. 1 is a representation of geometry and placement of a 3 x 3 convolution filter 102 in the art, centered on a pixel 103 in the upper left comer of a portion of a pixilated image 101. The rectangular geometry of filter 102 is seen to relate a central pixel 103 to its eight surrounding closest neighbors. Only a small number of pixels at exaggerated spacing distance is shown for image portion 101.

Fig. 2 is an enlarged view of the filter of Fig. 1, with associated values, to illustrate the computational procedure for asserting the filter at one position. In this representation each of the nine pixels associated in the 3 x 3 filter pattern has been given a lower-case letter, a through i. Pixel e in this case is the center pixel 103 for which the value will be changed in one assertion of the filter. Each of the nine positions in the

pattern has also been associated with a signed multiplier, which is +x for pixel e (103), and -1 for each of the surrounding pixels. These numbers are multipliers in an algorithm associated with the filter. Theoretically all of the multipliers may have unique values, but for reasons of efficacy and by experience -1 is an appropriate choice for the eight surrounding pixels. This multiplier is not the original pixel value at each position, but a

multiplier to be applied to the pixel value at that position each time the filter is asserted. The procedure for the filter we are considering operates as follows: (1) Determine the algebraic sum of the multipliers. This number is set aside as a divisor for step (3) below. In the example of Fig. 2 this divisor is -8+x.

-9(2) Multiply each pixel value by the assigned multiplier and take the algebraic sum of the results (-a-b-c-d+xe-f-g-h-i) (3) Divide this result by the divisor determined in step (1) above. (4) Replace the pixel value e by this new pixel value 5

(5) Move the filter to determine a new pixel value for another center pixel. Fig. 3 is a representation of how the filter of Fig. 1 might be moved over an image to produce heightened sharpness for the entire image. First the filter is applied at the upper left comer of the image (A), over a center pixel that is the second pixel in the second row. The filter procedure is applied for that center pixel, then the filter is moved

one pixel distance to the right (B), and applied to change the value for the third pixel in the second row. This move and calculate procedure is repeated for all the pixels in the second row, then the filter returns to the second column centered on the second pixel in the third row (C). Then the filter is moved sequentially one pixel at a time through the third row. This row-by-row procedure continues until all new pixel values are

determined that can be determined given the geometry of the filter. Now a new image is stored that has enhanced sharpness compared to the original image. The skilled artisan will recognize, of course, that the concept of a 3 x 3 filter with multipliers assigned to each cell in the filter is just a convenient concept. What actually happens is that a software routine, executing from a machine-readable medium coupled

to a computer appliance, consults a mapping of values in memory for an original image that is to be modified by the algorithm, selects the appropriate values that are associated with nine adjacent pixels, performs the steps of the algorithm, stores the new center pixel value in memory for a new image, and then selects a new mapping of nine adjacent pixels (moves the filter). If the pixel groups are selected in a manner that every pixel that may

be the center pixel of a 3 x 3 mapping, then new values will be determined a stored for all pixels in the original image, except edge pixels. Also a part of the algorithm is a step dictating that if a new value for a pixel is determined to be zero or less than zero, zero is used; and if a new value is determined that

- 10 is greater than the maximum allowed (255 for an eight-bit computing machine), then 255 is used. Given the procedure above for applying the convolution filter, it will be apparent to the skilled person that in step (1), if one chooses 9 for the multiplier in the center cell

(x), than the divisor for step (3) is 1, and in effect step (3) may be skipped; a bonus in

computation efficiency. It is rather well-known in the art that a 3 x 3 convolution filter with -1 as a

multiplier for the surrounding pixels is an appropriate choice to minimize computation intensity, but to provide differences in sharpness in an altered image from an original that 10

is not too extreme, it is necessary to use a multiplier x that will require a division step with a divisor greater than 1. Using 9 for x generates an amended image that has dramatic enhanced sharpness. To produce an image enhanced for sharpness by a JAVSD, it is necessary to use a much larger number for x. In practice it is seen that x needs to be about 18, producing a divisor of 10, to produce a sharpness-enhanced image

at JAVSD. A typical user, however, will not be satisfied with viewing just a first enhanced image. A user will typically want to see images enhanced step-by-step, until it is apparent the image is too sharp. Then the user may back down to the just previous image as the best choice. To accomplish this in the art, assuming that x=18 produces a JAVSD, requires

that an enhanced image be produced by applying the filer with x= 18 (divisor 10) at all positions that can be attained. Then a second enhanced image is produced with x= 17 (divisor 9), and so on (x = 16, 15, 14 ... ), until the user discovers the new image is too sharp. This may require four or more image iterations with a new divisor greater than 1 for each. The skilled person will understand that the original image needs to be saved,

and each enhanced image produced from the original or from a previously enhanced image also needs to be saved, and functionality needs to be provided for the user to select anyone of the images as the preferred image for sharpness.

- 11 -

The process described above for producing and displaying enhanced images to a viewer for selection to produce an image with preferred sharpness, is still quite computationally intensive. Fig. 4 illustrates a computerized appliance 401 having Internet connection via a 5

wireless network that communicates with a station 402, thence through a gateway 403 to Internet backbone 405, which represents all of the network interconnections in the Internet network. Two Internet-connected servers 406 and 407 are shown representing all of the sites in the Internet network which may serve information and data to internetconnected appliances like appliance 401. Appliance 401 may be a cellular telephone, a

personal digital assistant, or any other Internet connectable appliance. In some cases appliance 401 may be a laptop or desktop computer, or an iPad device. Internet connection is represented in Fig. 4 as a means by which appliance 401 may receive images, however, in some embodiments of the invention there may be no Internet connection, and images may be loaded to the appliance by any other known data transfer

technique. The skilled person will understand that appliance 401 will have a CPU and a display, and will be capable of executing software 408 stored in local memory, without this specification detailing the well-known components used in computerized appliances for displaying images, and for executing software that may alter pixel values and display

altered images from original images stored in memory coupled to the device. Fig. 5 illustrates a process for selectively enhancing sharpness of relatively large images in a minimum amount of time, using devices of limited computational power. It is well-known that the tendency in the art, due in part to the ever-descending cost of memory and greater resolution in displays, is to images of higher resolution. Given the

descriptions above regarding the computational intensity of sharpness enhancement, images of higher and higher resolution present an ever bigger problem in time and computing power. There are, however, quite good downsizing algorithms available to render high-resolution images at lower resolution. One site that deals with this issue is http-.~L-:~:}Y~Jl1~p-l1Q.!Q.t}!!i~h~~::u~Q!nLl~11l:gg~Y1l:p-;;.i?;~,.htm .

- 12 So a first step in sharpness enhancement in an embodiment of the present invention is to downsize the image desired to be enhanced. In Fig. 5 image 501 is an image for which a user desires enhanced sharpness. This image is represented as 2048 x 1536, which is 3,145,728, or 3.146 mega-pixels. This particular size is used only for 5

exemplary purpose, and could be any image of high-resolution. A first step is downsizing this image to a lower resolution, using a commercially available downsizing algorithm.

In this example image 501 is downsized to 480 x 320, or 153,600 pixels, about 5% of the number of pixels in the larger image. The skilled person will recognize that the 10

visual quality of downsized image 502 will be essentially the same as the image 501, as long as the display is presented without too much magnification. The smaller image will be quite satisfactory for a user to make judgments as to the quality of sharpness. A very big advantage is that application of a convolution filter to the smaller image will have to deal with only one pixel in twenty of the larger image, and can operate either twenty

times quicker, or with far less computing power in the same time frame. The next step in this unique process is running a minimal-computation convolution filter over the smaller image in several steps to create a series of altered images with just-appreciable visual difference from one image to the next, to create a series of enhanced images 503, 504, 505 506. There is a unique difference in the way

this is done in this example than in the prior art. In this embodiment the 3 x 3 filter with surrounding multipliers of -1 and x = 9 is used. this provides for the minimum computation, because the divisor for step (3) on page 5 above will be 1, which allows us to skip that step. The result of the single pass of the minimal convolution filter, however is that the

sharpness change will be quite dramatic, as described previously above, beyond what most users would select as a desirable improvement. In this embodiment this is handled in a unique way. The pixel vales for the original image are saved as image 0, and the first enhanced image is treated as image 10. Now our system takes, for each pixel, the difference between the pixel value for image 10 and the pixel value for image 0, and

- 13 -

divides by ten. These differences are algebraically added back to image 0 to produce image 1, an image with a just appreciable visual difference in sharpness from image O. Adding the differences to the pixel values for image 1 produces image 2, an image with a JAVSD from image 1, and a greater difference in sharpness from image O. The process is 5

repeated through image 9. Image 10 already exists as the result of applying the convolution filter to image O. We now have a series of ten images, each differing from its immediate neighbors by JAVSD. A user may easily select the image that appears to be the best (in the eye of the beholder) for sharpness.

It is not required that there be ten iterations. There may be five, or six, or four;

but there needs be several, so the user has a selection of several images from which to choose. If the selection is too sparse, the best image to the user might well be between two of the iterations presented. That is, one will appear to the user to be not sharp enough, and an adjacent iteration will appear too sharp. 15

Once the "best" image is chosen by the user, it is needed to provide that result to the larger resolution, which in this example is 2048 x 1536. In the prior art the process would dictate that the filter be applied to the larger image. But in this embodiment of the invention the best smaller image is simply upsized by a commercially-available algorithm that has been determined to be appropriate. The result has been shown by the inventor to

be equal in quality to the prior art method of applying more computation-intensive filters to the larger image, a process perhaps requiring orders of magnitude more power and time. Fig. 6 is a process flow diagram (flow chart) illustrating steps in a process according to a preferred embodiment of the present invention, much as described above.

At step 601 an image is selected to be enhanced for sharpness. At step 602 the image is downsized by applying a commercially-available downsizing algorithm, to a resolution substantially less than the resolution of the original image selected in step 601, providing an image 0 (502 in Fig. 5). At step 603 a convolution filter is applied to the reducedresolution image to form a first image with enhanced sharpness (image 1 - 503 in Fig. 5).

- 14 At step 604 the algebraic difference in value between the pixel values for image 1 and the original downsized image is determined. This algebraic difference is divided by an integer n in step 605. At step 606 the quotient for each pixel is added to the pixel value for image "0" to produce a second enhanced image 2. At step 607 the same quotient for 5

each pixel from step 605 is added to the pixels of image 2 to produce a third enhanced image 3. Step 707 is repeated to add values to pixels of image 3, and so on, until an image n-l is produced. At this point there are n enhanced images (0 to n), each displaying a JAVSD with the one before. These may be displayed to a user, preferably in order, and the user is invited to select the image judged to be the best for sharpness. This

is then upsized in the final act back to the original resolution of the larger image that was first considered to be enhanced for sharpness. This final image may be saved. The skilled person will recognize that the embodiments described herein may be altered in several ways within the scope of the invention. The size of the "larger" image is not a fixed value, for example, but can be anyone of a wide variety of resolutions. The

downsizing and up sizing algorithms are not fixed, but may be chosen from a variety of readily-available and well-known algorithms. The size of the "smaller" image is not fixed either, but may vary over a wide range. The smaller image is preferably considerably smaller than the larger to effectively limit the number of pixels necessary to recalculate in filter application. The number of iterations from image 0 to a final image,

each of which is produced by a single pass of the filter, is also not fixed, but is preferably at least four, and more preferably eight or ten. Another variation in the process might involve producing one alteration at a time, and allowing the user to judge the new image before going on to a next. For example, the system might present the first alteration to the user and wait for a signal to produce the

next, and then wait for a signal to produce another. The user may have access as well to a "back" command, and to a command to compare the image with the original, so when an image is presented that is slightly too sharp, the back command will revert to the just previous image, and the user may then cause that image to be selected and upsized to the original resolution. There are many similar possibilities.

- 15 -

Batch Processing In another aspect of the present invention it may be desirable to do batch processing, that is, to apply the process described above in different examples to a 5

plurality of digital images. A user may have, for example, a plurality of images of very similar characteristics, such as a group of images captured by a digital camera in a relatively short period of time, under similar circumstances of lighting, and without changing settings on the camera, and displayed on the same monitor, perhaps a computer display monitor.

Referring now to Fig. 6 and to the description of Fig. 6 above, and not considering the downsizing or upsizing of an image, a process is described wherein a minimal convolution filter is sequentially applied over the pixels of an image to be enhanced (image 0), producing pixel values for an image "n". Sharpness is thus enhanced for image "n", but the change (increase) in sharpness will typically be more than might be

desired. In this process the algebraic difference between the value for each original pixel and the value for the pixel in the same position in the image for image "n" is determined, and then the difference is divided by an integer. The integer may be theoretically any integer, but the idea is to produce images between image 0 and image "n", in which there is just an appreciable visual difference. So this integer value may be 10, for example, and

10 is set as "n". The pixel value differences at each pixel position is divided by the integer, then an image 1 is produced by adding to the pixel values at each pixel position for image 0 one-tenth of the difference between the pixel value for image 0 and the pixel value for the same pixel in image 10. Similarly an image 2 is produced adding .2 times the difference

at each pixel position, an image 3 using .3 times the difference, and so on, producing images 1 through 9 between image 0 and image 10, each successive image having a sharpness increase of just an appreciable visual difference. These ten images are displayed to a user, the user enabled to select the "best" image, that is, the one that seems to have, for that user, the optimum quality of sharpness.

- 16 Assume now that this unique process is followed for one image of a plurality of images of very similar characteristics by a user, and the user selects image 3 as the optimum image sharpness. It may be assumed, then, that the image 3 for all of the plurality of images will be, for this user, the image with the optimum sharpness. It will 5

not be necessary to produce image 1, 2, or 4 through 9 for any of the other images of the plurality. Having selected the plurality of images and initiated a batch process, the batch system in this embodiment will produce the ten images for the user, and enable selection of the "best" image for the user, and then use the image selected (1st, 2nd, 3rd, etc.), to produce a sharpness-enhanced image for all of the other images of the plurality.

Fig. 7 is a process flow diagram for the batch processing process described just above, in which steps 702 through 708 repeat the process described with reference to Fig. 6 for one image. Step 701 is a first step for selecting the first image to be enhanced from a plurality of similar images. Step 709 is for noting the number of the image chosen by the user as the optimum image for sharpness, and step 710 repeats the process for every

other image of the plurality, but to produce only the image of the number selected. So it is only necessary to produce all of the images for selection just once, then all of the other enhanced images may be produced automatically. Another example of batch processing in sharpness enhancement is in the area of video technology. It is well-know that data streams for video are arranged to produce

successive frames in display, much in the manner of movie film presenting a rapidlychanging sequence of still images, each slightly altered from the previous. Typically all of the frames in a video data stream will have very similar sharpness characteristics. If one selects, then, just one frame, applies the process described above to the one frame, and a user selects one of the candidate sharpness-enhanced images as the most

appropriate, then further processing may be truncated for all the other frames. Assume, for example, that the process is applied to one frame, providing ten candidate images of that frame, each with a JAVSD from the immediately preceding image, and the user selects image four. One may safely assume that image four will be appropriate for all the other frames of the video as well. and the original image for each frame may then be

- 17 processed to produce the fourth image, without producing all of the other candidate Images.

Selective Segmentation in Image Sharpening 5

The processes described thus far in this specification apply the same process in sharpness enhancement to every part of an image. The skilled artisan will understand that in some cases a user may prefer to preferentially enhance sharpness in particular segments of an image. In one instance, for example, a user may prefer to sharpen an image preferentially in segments that are in shadow. In another instance a user may

prefer to sharpen an image preferentially in segments that are highlighted, that is in brightness. In yet another instance a user may prefer to sharpen an image preferentially in segments that are in midrange of brightness. Other similar preferences are possible. Therefore, in another aspect of the present invention, a process is provided that treats different segments of an image differently, according the local brightness

characteristics. Fig. 8 is a diagram that relates average local pixel value from 0 to 255 (assuming an eight-bit display system) for brightness to a vertical scale between 0 (at origin) and 1. In Fig. 8 there is a straight line labeled S for use in a process to sharpen preferentially in segments that are in shadow. A straight line labeled H is for use in a process to sharpen preferentially in segments of an image highlighted. A curve M is for

use in a process to sharpen preferentially in midtone segments. Assume for a first example that a user wants to sharpen preferentially in segments that are in shadow. The process is very similar to that described above as conventional art for producing an image n from an original image 0, in which a convolution filter is applied sequentially to values for pixels of image 0 to produce image n. In this new and

non-conventional procedure, however, at each application of the filter, in addition to producing a new pixel value for image n, the pixel values for each cell of the filter (nine cells for a three by three filter) are added and divided by the number of the cells, rendering a pixel value average for the cells in the vicinity of the center cell being altered

- 18 in value by the filter protocol. This average pixel value expresses the nature of the segment in which the object pixel resides; that is, light, dark or midtone. Now, in this preferential process for sharpening in shadow, the system utilizes the graph of Fig. 8, and line S. Note that for a local pixel value average of zero (fully dark, 5

extreme shadow) the graph expresses 1, and for a local pixel value average of255 (fully bright) the graph expresses zero, and for local averages in between, the value from the graph is between 0 and 1 proportionally. At every position for application of the filter the local average pixel value is determined, and the new pixel value determined by the filter protocol is used to

determine the pixel value difference for that pixel position, and the difference is multiplied by the value from the graph and added back for the new pixel value. It will be apparent to the skilled person that this procedure, using the line S, will

preferentially sharpen in segments that a dark, and will sharpen less in segments that are more light. 15

Ifit is desired to sharpen preferentially in areas that are light, then in the process the line H is used from the graph of Fig. 8, and sharpness will be preferential for light segments, and less for dark segments. If it is desired to sharpen in mid-tone segments, the curve M will be used, so maximum effect will accrue for local averages near midtone (halfway between 0 and 255), and there will be little effect near deep shadow or extreme

brightness, and the effect will demonstrate uniform differences, because the curves are well-behaved. It will be apparent to the skilled person that a graph may be created for just about

any segment enhancement. M may be inverted, for example, to sharpen preferentially in both deep shadow or extreme brightness, but not at all at mid-tone. The skilled person 25

will also understand that the graph of Fig. 8 is used to illustrate the process, but that in practice values will be picked from tables relating local pixel value averages to decimal fractions between 0 and 1. Fig. 9 is a flow diagram depicting the process just described for preferential sharpening in segments based on local pixel values averages indicating relative lightness

- 19 or darkness in the image in the local vicinity. At step 901 the convolution filter is applied for a first pixel in the original image "0". At step 902 the algebraic difference between the original pixel value at the first position and the new pixel value is determined. At step 903 a multiplier is selected according to the graph using the relationship for a 5

segment preference. At step 904 the difference from step 902 is multiplied by the multiplier from step 903. At step 905 the result of step 904 is added back to the original pixel value. At step 906 the result of step 905 is saved as the new pixel value for image "n". At step 907 the filter is moved to a new position and the process is repeated until new pixel values are determined for all of the pixel positions.

It will be apparent to the skilled person that there are many alterations that might

be made to embodiments described as examples herein, all within the scope of the invention, which is limited only by the claims that follow.

- 20-

CLAIMS

1. A method for enhancing sharpness for a digital image, comprising the steps of: 5

(a) in a display of a computerized appliance, selecting an image to be enhanced in sharpness; (b) downsizing the selected image by a downsizing algorithm executing on the computerized appliance to produce an image 0 at resolution substantially less than resolution of the original image selected in step (a);

(c) applying a convolution filter to image 0 to produce an image n with enhanced sharpness, where n is an integer; (d) subtracting pixel values for pixels of image n from corresponding pixels for image 0, saving the differences; (e) dividing the differences in step (d) by integer n, and saving the quotients;

(f) adding the quotients from step (e) to values for corresponding pixels in image

oto produce an image 1, then to values of pixels for image 1 to produce an image 2, and repeating until an image n-l is produced; (g) presenting images 0 through n to a user for selection of a best image for sharpness; and 20

(h) upsizing the user-selected image by an up sizing algorithm back to the resolution of the image selected in step (a).

2. The method of claim 1 wherein the convolution filter is a 3 x 3 filter with multipliers of -1 at all cells but the center cell. 25 3. The method of claim 2 wherein the multiplier at the center cell is 9, producing a divisor of 1 for application of the filter.

4. The method of claim 1 wherein n = 10.

- 21 -

5. A system for enhancing sharpness for a digital image, comprising: a computerized appliance having a digital display and executing software from a machine-readable medium, the software providing: 5

a mechanism enabling a user to select an image to be enhanced; a downsizing algorithm enabling the user to downsize the selected image to a resolution substantially less than the than resolution of the original image selected; a convolution filter and functions for applying the convolution filter to stored images to produce images enhanced for sharpness; and

an upsizing algorithm enabling the user to upsize an image to a higher resolution; wherein the user selects an image to be enhanced in sharpness, the image is downsized to produce an image 0 at resolution substantially less than resolution of the original image selected, the convolution filter is applied to image 0 to produce an image n with enhanced sharpness, where n is an integer, the pixel values for pixels of image n are

subtracted from corresponding pixels for image 0, saving the differences, the differences are divided by n, saving the quotients, the quotients are added back to the pixel values for image 0 to produce an image 1, and the process is repeated adding the quotients to pixel values of image 1 to produce an image 2, and so forth, until an image n-l is produced, then images 0 through n are presented to the user for selection of a best image for

sharpness, then the selected image is upsized back to the resolution of the original image selected to be enhanced in sharpness.

6. The system of claim 5 wherein the convolution filter is a 3 x 3 filter with multipliers of -1 at all cells but the center cell. 25 7. The system of claim 6 wherein the multiplier at the center cell is 9, producing a divisor of 1 for application of the filter.

8. The system of claim 1 wherein n = 10.

- 22-

9. A method for enhancing sharpness for a digital image, comprising the steps of: (a) in a display of a computerized appliance, selecting an image to be enhanced in sharpness; 5

(b) downsizing the selected image by a downsizing algorithm executing on the computerized appliance to produce an image 0 at resolution substantially less than resolution of the original image selected in step (a); (c) applying a convolution filter to image 0 to produce an image n with enhanced sharpness, where n is an integer;

(d) subtracting pixel values for pixels of image n from corresponding pixels for image 0, saving the differences; (e) dividing the differences in step (d) by integer n, and saving the quotients; (f) adding the quotients from step (e) to values for corresponding pixels in image

oto produce an image 1; 15

(g) displaying image 1 to a user and asking for approval; (h) in case of no approval at step (g), adding the quotients from step (e) to the pixel values for image 1 to produce an image 2; (i) repeating building new images by process of steps (g) and (h) until the user selects one as best image; and

(j) upsizing the user-selected image by an up sizing algorithm back to the resolution of the image selected in step (a).

10. The method of claim 9 wherein the convolution filter is a 3 x 3 filter with multipliers of -1 at all cells but the center cell. 25 11. The method of claim 10 wherein the multiplier at the center cell is 9, producing a divisor of 1 for application of the filter.

12. The method of claim 9 wherein n = 10.

- 23 -

13. A system for enhancing sharpness for a digital image, comprising: a computerized appliance executing software from a machine-readable medium, the software providing: 5

a mechanism enabling a user to select an image to be enhanced; a downsizing algorithm enabling the user to downsize the selected image to a resolution substantially less than the than resolution of the original image selected; a convolution filter and controls for applying the convolution filter to stored images to produce images enhanced for sharpness; and

subtracted from corresponding pixels for image 0, saving the differences, the differences are divided by n, saving the quotients, the quotients are added back to the pixel values for image 0 to produce an image 1, image 1 is displayed to the user to approve or not as a best image for sharpness, in the case of no approval the saved quotients are added to the pixel values of image 1 to produce an image 2, which is displayed to the user for

approval, and the process is repeated until the user selects an image as the best image for sharpness, then the selected image is upsized back to the resolution of the original image selected to be enhanced in sharpness.

14. The system of claim 13 wherein the convolution filter is a 3 x 3 filter with multipliers 25

of -1 at all cells but the center cell.

15. The system of claim 14 wherein the multiplier at the center cell is 9, producing a divisor of 1 for application of the filter.

- 2416. The system of claim 13 wherein n = 10.

17. A method for producing a sequence of images enhanced for sharpness, comprising the steps of: 5

(a) selecting an image to be enhanced as image 0; (b) applying a convolution filter to image 0 to produce an image n with enhanced sharpness, where n is an integer; (c) subtracting pixel values for pixels of image n from corresponding pixels for image 0, saving the differences;

(d) dividing the differences in step (c) by integer n, and saving the quotients; and (e) adding the quotients from step (d) to values for corresponding pixels in image

oto produce an image 1, then to values of pixels for image 1 to produce an image 2, and repeating until an image n-l is produced.

18. The method of claim 17 comprising a further step for presenting the images as a sequence of images to a user for selection of one of the images as a best image for sharpness.

- 25 ABSTRACT OF THE DISCLOSURE

A method for enhancing sharpness for a digital image follows this sequence: (a) in a display of a computerized appliance, selecting an image to be enhanced in 5

sharpness; (b) downsizing the selected image by a downsizing algorithm executing on the computerized appliance to produce an image 0 at resolution substantially less than resolution of the original image selected in step (a); (c) applying a convolution filter to image 0 to produce an image n with enhanced sharpness, where n is an integer; (d) subtracting pixel values for pixels of image n from corresponding pixels for image 0,

saving the differences; (e) dividing the differences in step (d) by integer n, and saving the quotients; (f) adding the quotients from step (e) to values for corresponding pixels in image 0 to produce an image 1, then to values of pixels for image 1 to produce an image 2, and repeating until an image n-l is produced; (g) presenting images 0 through n to a user for selection of a best image for sharpness; and (h) upsizing the user-selected image

by an upsizing algorithm back to the resolution of the image selected in step (a).

Electronic Patent Application Fee Transmittal Application Number: Filing Date:

Title of Invention:

Sharpness in Digital Images

First Named Inventor/Applicant Name:

Rodney Shaw

Filer:

Donald Rex Boys/Sheri Beasley

Attorney Docket Number:

P882

Filed as Small Entity

Utility under 35 USC 111 (a) Filing Fees Description

Fee Code

Quantity

Amount

Sub-Total in USD($)

Utility filing Fee (Electronic filing)

4011

Utility Search Fee

2111

270

Utility Examination Fee

2311

110

Independent claims in excess of 3

2201

110

220

Basic Filing:

Pages:

Claims:

Miscellaneous-Filing:

Petition:

Description

Fee Code

Quantity

Amount

Sub-Total in USD($)

Patent-Appeals-and-Interference: Post-Allowance-and-Post-Issuance: Extension-of-Time: Miscellaneous:

Total in USD ($)

682

Electronic Acknowledgement Receipt EFSID:

8661904

Application Number:

12908161

International Application Number: Confirmation Number:

3367

Title of Invention:

Sharpness in Digital Images

First Named Inventor/Applicant Name:

Rodney Shaw

Customer Number:

24739

Filer:

Donald Rex Boys/Sheri Beasley

Filer Authorized By:

Donald Rex Boys

Attorney Docket Number:

P882 20-0CT-2010

Receipt Date: Filing Date: TimeStamp:

12:59:04 Utility under 35 USC 111 (a)

Application Type:

Payment information: yes

Submitted with Payment Payment Type

Deposit Account

Payment was successfully received in RAM

$682

RAM confirmation Number

9417

Deposit Account

500534

Authorized User

File Listing: Document Number

Document Description

File Name

File Size(Bytes)/ I Multi Message Digest Part /.zip

Pages (ifappl.)

1347662 1

Application Data Sheet

US_ADSJorm_SB_14a.pdf

yes

c7971 e9fOfaOc83791fddOc026d40016164d bSf1

Warnings: Information: 63858 2

Oath or Declaration filed

POA882.pdf Ob9fa 1621 fd40468c987f14e3de664f2fd62 eOO

Warnings: Information:

317943

Drawings-only black and white line drawings

Drawings.pdf 8e 1b2 6b8a803 6ac4d Sa 7 5 03 bfOa 75 21289f 01321

Warnings: Information: 156533 4

P882PA.pdf 44e37a7971 ed4ccc69ae201 dgec3923aa61 07078

Multipart Description/PDF files in .zip description Document Description

Start

End

Specification

Claims

Abstract

Warnings: Information: 35898 5

Fee Worksheet (PTO-875)

fee-info.pdf

no 40b9743f03edfb668cbf343835eaed938a7d cf16

Warnings: Information: Total Files Size (in bytes)

1921894

This Acknowledgement Receipt evidences receipt on the noted date by the USPTO of the indicated documents, characterized by the applicant, and including page counts, where applicable. It serves as evidence of receipt similar to a Post Card, as described in MPEP 503. New Applications Under 35 U.S.c. 111 If a new application is being filed and the application includes the necessary components for a filing date (see 37 CFR 1.53(b)-(d) and MPEP 506), a Filing Receipt (37 CFR 1.54) will be issued in due course and the date shown on this Acknowledgement Receipt will establish the filing date of the application. National Stage of an International Application under 35 U.S.c. 371 If a timely submission to enter the national stage of an international application is compliant with the conditions of 35 U.S.c. 371 and other applicable requirements a Form PCT/DO/EO/903 indicating acceptance of the application as a national stage submission under 35 U.S.c. 371 will be issued in addition to the Filing Receipt, in due course. New International Application Filed with the USPTO as a Receiving Office If a new international application is being filed and the international application includes the necessary components for an international filing date (see PCT Article 11 and MPEP 1810), a Notification of the International Application Number and of the International Filing Date (Form PCT/RO/l 05) will be issued in due course, subject to prescriptions concerning national security, and the date shown on this Acknowledgement Receipt will establish the international filing date of the application.

PTO/SB/06 (12-04) Approved for use through 713112006. OMB 0651-0032 U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE Under the Paperwork Reduction Act of 1995 no persons are required to respond to a collection of information unless it displays a valid OMB control number , Application or Docket Number PATENT APPLICATION FEE DETERMINATION RECORD Substitute for Form PTO-875 .

Date:

10/20/10

12/908,161

APPLICATION AS FILED - PART I

FOR BASIC FEE (37 CFR 1.16(a), (b), or(c» SEARCH FEE (37 CFR 1.16(k), (i), or (m» EXAMINATION FEE (37 CFR 1.16(0), (P), or (q» TOTAL CLAIMS (37 CFR 1.16(i)) INDEPENDENT CLAIMS (37 CFR 1.16(h» APPLICATION SIZE FEE (37 CFR 1.16(5»

(Column 2)

NUMBER FILED

NUMBER EXTRA

RATE ($)

FEE ($)

RATE ($)

N/A

270

N/A

110

N/A

minus 20 =

SMALL ENTITY

OTHER THAN SMALL ENTITY

(Column 1)

X$26=

X$110=

minus 3 = If the specification and drawings exceed 100 sheets of paper, the application size fee due is $270 ($135 for small entity) for each additional 50 sheets or fraction thereof. See 35 U.S.C. 41 (a)(l)(G) and 37 CFR

X$52

220

X$220=

N/A

MULTIPLE DEPENDENT CLAIM PRESENT (37 CFR 1.160»

N/A

TOTAL

* If the difference in column 1 is less than zero, enter "0" in column 2.

682

TOTAL

APPLICATION AS AMENDED - PART II

c:(

I-

:I!! C Z

:I!!

c:(

(Column 1) CLAIMS REMAINING AFTER AMENDMENT

(Column 3)

(Column 2) HIGHEST NUMBER PREVIOUSLY PAID FOR

Total * Minus ** (37 CFR 1.16(i)) Independent Minus *** (37 CFR 1.16(h» * Application Size Fee (37 CFR 1.16(s»

SMALL ENTITY

PRESENT EXTRA

RATE ($)

FIRST PRESENTATION OF MULTIPLE DEPENDENT CLAIM (37 CFR 1.160))

III IZ

:I!!

C Z

:I!! c:(

(Column 2)

(Column 3)

CLAIMS REMAINING AFTER AMENDMENT

HIGHEST NUMBER PREVIOUSLY PAID FOR

PRESENT EXTRA

Total Minus ** * (37 CFR 1.16(i)) Inaependent Minus *** * (37 CFR 1.16(h)) Application Size Fee (37 CFR 1.16(s»

RATE ($) OR

N/A

ADDITIONAL FEE ($)

= =

X X

N/A

TOTAL ADD'T FEE

(Column 1)

OTHER THAN SMALL ENTITY

ADDITIONAL FEE ($)

= =

FEE ($)

TOTAL ADD'T FEE

OR RATE ($)

FIRST. PRESENTATION OF MULTIPLE DEPENDENT CLAIM (37 CFR 1.16(j))

= = N/A

TOTAL ADD'T FEE

ADDITIONAl FEE ($)

RATE ($) OR

OR OR OR

ADDITIONAL FEE ($)

= =

X X

N/A TOTAL ADD'T FEE

* If the entry in column 1 is less than the entry in column 2, write 'a' in column 3. *. If the 'Highest Number Previously Paid For" IN THIS SPACE is less than 20, enter '20'.

...

If the 'Highest Number Previously Paid For" IN THIS SPACE is less than 3, enter '3' . The 'Highest Number Previously Paid For" (Total or Independent) is the highest number found in the appropriate box in column 1. ThiS collection of Information IS reqUIred by 37 CFR 1.16. The information IS reqUIred to obtain or retain a benefit by the public which IS to file (and by the USPTO to process) an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.14. This collection is estimated to take 12 minutes to complete, including gathering, preparing, and submitting the completed application form to the USPTO. Time will vary depending upon the individual case. Any comments on the amount of time you require to complete this form and/or suggestions for reducing this burden, should be sent to the Chief Information Officer, U.S. Patent and Trademark Office, U.S. Department of Commerce, P.O. Box 1450, Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. SEND TO: Commissioner for Patents, P.O. Box 1450. Alexandria, VA 22313·1450. If you need assistance in completing the form. call1·80()''PTO·9199 and select option 2.