Camera SDK for iOS

Integrate the Camera SDK

1. Download SDK

SDK: iOS_SDK_v3.1.11Swift Latest

SDK Use Demo: iOS SDK Demo Source Code  Latest

Release Notes: Changelog New

All Versions: Releases

2. Put SDK to project

Unarchive the SDK archive file and place StitchingCameraSDK.xcframework to Project

3. Add dependencies

• In XCode TARGETS -> General -> Frameworks,Libraries,and Embedded Content,click +, Choose the SDK framework

• StitchingCameraSDK.xcframework is Dynamic Library Framework, Embeb item Choose Embed & Sign

To understand how to write the business logic that deals with Camera SDK interactions, please refer to the following sections.

Refer SDK Use Demo Project

The above steps guide you through the basic integration process. To learn how to interact with the Camera SDK, please unarchive the SDK Use Demo archive and review the detailed code.

The following sections provide a brief introduction to the core components.

1. Initialize callback for Camera SDK and Register

//StitchingCameraCallback interface function definition
//Sample code to temporarily manage image information, you can define it by yourself
var imagesCached:[Any] = [Any]()

/**
 * Callback after taking a single image
 * @param imagePath Single Image local save path
 * @param imageIndex image number
 * @param imageId mage unique encoding UUID
 * @param pair The information associated with the previous puzzle (only classic stitching method will have this information)

*/
func takeSinglePhotoCallback(_ imagePath: String, _ imageIndex: Int, _ imageId: String, _ pair: String) {
    // Sample code Images Cache Manager
    self.imagesCached.append([imagePath, imageIndex, imageId, pair])

    // Sort the list of imagesCached by imageIndex in ascending order
    self.imagesCached.sort { (($0 as? [Any])?[1] as? Int) ?? 0 < (($1 as? [Any])?[1] as? Int) ?? 0 }
}

/**
* The callback when end taking the photo(s) task
* 
* @param stitchingPath stitching local path
* @param stitching stitching info
*    stitchingInfoVersion 0 (classic stitching mode), 1 (video stitching mode), 3 (ultimate stitching mode)
*    info puzzle information (video stitching mode will have this information)
*    BestPov latest stitching position (only class stitching mode has this information)
*    resize puzzle compression ratio (only the class stitching mode has this information)
* */
func endTakePhotoCallback(_ stitchingPath: String, _ stitching: String) {
    // stitch return has the following two situations
    // Situation 1: The single image of this group of stitching tasks has only one image, so there is no need to initiate a stitch task request to the server, and the 'task_id' returned by the single image is used to obtain the recognition result
    // Situation 2: If there are more than one picture in this group of stitching tasks, it is nead to initiate a [stitch request to the server](https://retail-doc.clobotics.com/en/open-api/api/ir-api.html#create-stitching-task)

    // jsonString to Dic
    let stitchingJSONObj = toDictFromJSONString(jsonString:stitching)
    let stitchingCode:Int = stitchingJSONObj["stitchingInfoVersion"] as! Int
    if (stitchingCode == 0) {
        // Please refer to `StitchingInfo management` -> `video stitching mode` method to generate
    }
    if (stitchingCode == 1) {
        // Please refer to `StitchingInfo management` -> `classic stitching mode` method to generate
    }
    if (stitchingCode == 3) {
        // Please refer to `StitchingInfo management` -> `ultimate stitching mode` method to generate
    }
}

/**
* The callback when cancel taking the photo(s) task
* 
* @param single image local path Array need to remove
* */
func cancelTakePhotoCallback(_ imagesPath: [String]) {
    /** your code here */
}

/**
 * It will call by switching stitching modes(classic mode and video mode), or giving up taking photos.
 */
func clearAll() {
    // Sample code
    self.imagesCached = [Any]()
}

- (void)takeSinglePhotoCallback:(NSString *)imagePath :(NSInteger)imageIndex :(NSString *)imageId :(NSString *)pair {
    /** your code here */
}
- (void)endTakePhotoCallback:(NSString *)stitchingPath :(NSString *)stitching {
    /** your code here */
}
- (void)cancelTakePhotoCallback:(NSArray<NSString *> *)imagesPath {
   /** your code here */
}
- (void)clearAll {
   /** your code here */
}

In the StitchingCameraCallback interface, there are several important callbacks that need attention:

• delPhotoCallback: It is invoked when the user cancels a photo during shooting. It's crucial to handle this callback logic, especially in stitching mode, to prevent incorrect StitchingInfo.

/**
 * try to remove the last photo callback 
 * @param imagePath image save path
 */
func delPhotoCallback(_ imagePath:String) {
    // your code here trys to remove imapgePath from imagesCached
}

- (void)delPhotoCallback:(NSString *)imagePath {
   
}

2. Initialize the CameraConfig and Launch Camera Instance

For details on setting CameraConfig, please refer to: Setup CameraConfig. Then, use the following code to launch the camera instance.

let vc = ParentStitchController()
//Set the callback for monitoring the camera event
 vc.mStitchingCallback = self 
 vc.modalPresentationStyle = .fullScreen
// CameraConfig Camera parameter configuration description For specific attributes, 
// please refer to the Parameters introduction for config
 vc.cameraConfig = mCameraConfig
 // open camera
 self.present(vc, animated: true)

ParentStitchController *vc = [[ParentStitchController alloc] init];
//Set the callback for monitoring the camera event
vc.mStitchingCallback = self;
vc.modalPresentationStyle = UIModalPresentationFullScreen;
// CameraConfig Camera parameter configuration description For specific attributes, 
// please refer to the Parameters introduction for config
vc.cameraConfig = config;
 // open camera
[self presentViewController:vc animated:YES completion:nil];

3. StitchingInfo Management

The crucial part of capturing stitching images is properly managing the StitchingInfo. If not managed correctly, when calling the Create Stitching Picture Task in the OpenAPI system, the task may fail due to incorrect information. Please read StitchingInfo Management carefully for detailed information.

Setup CameraConfig

1. Stitching Modes

Classic Stitching Mode

let mCameraConfig = CameraConfig()
mCameraConfig.userSelectedStitchingMode = 2
mCameraConfig.maskStyle = 2   //  can be omitted
mCameraConfig.stitchingMinCount = 2
mCameraConfig.stitchingMaxCount = 4

CameraConfig *mCameraConfig = [[CameraConfig alloc] init];
mCameraConfig.userSelectedStitchingMode = 2;
mCameraConfig.maskStyle = 2;   //  can be omitted
mCameraConfig.stitchingMinCount = 2;
mCameraConfig.stitchingMaxCount = 4;

And then Launch Camera

Notes:

• For the stitchingMinCount and stitchingMaxCount parameters, set them to the appropriate values based on your requirements.

Video Stitching Mode

let mCameraConfig = CameraConfig()
mCameraConfig.userSelectedStitchingMode = 1
mCameraConfig.stitchingMinCount = 2
mCameraConfig.stitchingMaxCount = 4

CameraConfig *mCameraConfig = [[CameraConfig alloc] init];
mCameraConfig.userSelectedStitchingMode = 1;
mCameraConfig.stitchingMinCount = 2;
mCameraConfig.stitchingMaxCount = 4;

And then Launch Camera

Notes:

• For the stitchingMinCount and stitchingMaxCount parameters, set them to the appropriate values based on your requirements.

Ultimate Stitching Mode

let mCameraConfig = CameraConfig()
mCameraConfig.userSelectedStitchingMode = 3
mCameraConfig.stitchingMinCount = 2
mCameraConfig.maxRecordingSecs = 45

CameraConfig *mCameraConfig = [[CameraConfig alloc] init];
mCameraConfig.userSelectedStitchingMode = 3;
mCameraConfig.stitchingMinCount = 2;
mCameraConfig.maxRecordingSecs = 45;

And then Launch Camera

Notes:

• For the stitchingMinCount and maxRecordingSecs parameters, set them to the appropriate values based on your requirements.

2. Single Capture Modes

Classic Single Capture Mode

In this mode, users can still capture multiple images simultaneously without exiting the camera. Unlike stitching mode, although multiple images are captured, the Camera SDK will not provide StitchingInfo.

let mCameraConfig = CameraConfig()

mCameraConfig.userSelectedStitchingMode = 2
mCameraConfig.maskStyle = 1

CameraConfig *mCameraConfig = [[CameraConfig alloc] init];
mCameraConfig.userSelectedStitchingMode = 2
mCameraConfig.maskStyle = 1

And then Launch Camera

Notes:

• For the stitchingMinCount and stitchingMaxCount parameters, set them to the appropriate values based on your requirements.

Pure Price-tag Capture Mode

This mode introduces a guide mask for scenarios involving the capture of price tags, which is a particular scenario of Classic Stitching Mode. In this mode, users can capture multiple single images with the mask displayed simultaneously without exiting the camera. The significant difference in this mode is that, although there are multiple images, Camera SDK won't provide StitchingInfo, and you also don't need to manage the StitchingInfo.

let mCameraConfig = CameraConfig()
/**
 * Pure Price-tag Mode 一 single picture only, other params in default.
 * */
mCameraConfig.userSelectedStitchingMode = 2
mCameraConfig.maskStyle = 3
mCameraConfig.stitchingMinCount = 1
mCameraConfig.stitchingMaxCount = 1

CameraConfig *mCameraConfig = [[CameraConfig alloc] init];
mCameraConfig.userSelectedStitchingMode = 2
mCameraConfig.maskStyle = 3
mCameraConfig.stitchingMinCount = 1
mCameraConfig.stitchingMaxCount = 1

Product Price-tag Capture Mode

This mode introduces a guide mask for scenarios involving the capture of product with price tag displayed on shelf, which is a particular scenario of Classic Stitching Mode. In this mode, users can capture multiple single images with the mask displayed simultaneously without exiting the camera. The significant difference in this mode is that, although there are multiple images, Camera SDK won't provide StitchingInfo, and you also don't need to manage the StitchingInfo.

let mCameraConfig = CameraConfig()
/**
 * Product Price-tag Mode 一 single picture only, other params in default.
 * */
mCameraConfig.userSelectedStitchingMode = 2
mCameraConfig.maskStyle = 4
mCameraConfig.stitchingMinCount = 1
mCameraConfig.stitchingMaxCount = 1

CameraConfig *mCameraConfig = [[CameraConfig alloc] init];
mCameraConfig.userSelectedStitchingMode = 2
mCameraConfig.maskStyle = 4
mCameraConfig.stitchingMinCount = 1
mCameraConfig.stitchingMaxCount = 1

Multi-language Support

Supported languages include English by default, with others available if specified.

Language	Parameter Value
Chinese	zh-Hans
Chinese Traditional	zh-Hant
English	en
Thai	th
Myanmar	my-MM
French	fr
Portuguese	pt
Spanish	es
Italy	it

Use CameraConfig class language properties set

let mCameraConfig = CameraConfig()
mCameraConfig.language = "zh-Hans"

CameraConfig *mCameraConfig = [[CameraConfig alloc] init];
mCameraConfig.language = "zh-Hans";

Attributes of CameraConfig class

Parts	Parameter	Description	Type	Default	Version
Multi-language Support	language	Set the display language. Supported languages	string	Follow system	3.1.6
Stitching	userSelectedStitchingMode	capture mode in default when starts, 1 means video stitching, 2 means classic capture mode(aligned with maskStyle assigned can specify stiching capture, single image capture or price-tag capture mode), 3 means ultimate stitching Note: when 1 assigned，needs hardware support, then video stitching would take effect	int	2	3.0.0
	maskStyle	Enable different styles overlay when taking picture: 1 means no guide frame (normal capture); 3 means pure price-tag guide frame; 4 means product price-tag guide frame Note: When configured as 1, 3, or 4, userSelectedStitchingMode = 2 is required to take effect.	int	0	3.1.5
	isFirstUse	Enable first-time usage guide or not	boolean	false	3.0.0
	allowRoll	Allow the user to rotate the angle along the Roll direction(0~90°)	float	45	3.0.0
	allowPitch	Allow the user to swing the angle along the Pitch direction(0~90°)	float	45	3.0.0
	stitchingMinCount	The minimum photo count for take a stitching task, 0 means not limit	int	0	3.0.0
	stitchingMaxCount	The maximum photo count for take a stitching task, 0 means not limit	int	0	3.0.0
	maxRecordingSecs	The maximum seconds for recording Note: userSelectedStitchingMode = 3 is required to take effect.	int	0	3.1.10
	enableGEOLocator	Enable embedding longitude and latitude coordinates into the EXIF data of photo or not	boolean	false	3.1.4
Image Quality Check	isUseLargeAngleModel	Enable large angle detection Note: when stitching mode is classic stitching (userSelectedStitchingMode = 2 && maskStyle = 2) or ultimate stitching(userSelectedStitchingMode = 3), isUseLargeAngleModel would take effect. Single Image Capture Modes won't take effect.	boolean	false	3.1.11
	allowLargeAngle	The maximum large angle when detection enabled(0~90°) Note: isUseLargeAngleModel=true is required to take effect.	float	20.0	3.1.10
	isUseSkuSizeModel	Enable check SKU-part's proportion of single picture（if enabled 30% in default） Note: only when stitching mode is classic stitching, isUseSkuSizeModel would take effect	boolean	false	3.0.0
	minSkuSizeRatio	The minimum of SKU size ratio required(0.0~1.0) Note: isUseSkuSizeModel=true is required to take effect.	float	0.3	3.1.10

StitchingInfo management

In Classic Stitching Mode

The following logic must be added to the func endTakePhotoCallback(_ stitchingPath: String, _ stitching: String) method, as shown in the SDK Use Demo project.

/**
 * in function endTakePhotoCallback definition
 * @param stitchingPath stitching local path
 * @param stitchingInfo stitching info
 * @param stitchingInfoVersion 0(classic stitching mode)、1(video stitching mode)、3(ultimate stitching mode)； 
 * info puzzle information (video stitching mode will have this information)
 * BestPov latest stitching position (only class stitching mode has this information)
 * resize puzzle compression ratio (only the class stitching mode has this information) 
 * */

let info = stitchingInfo.data(using: String.Encoding.utf8)
var uploadStitchingInfo = ""
if let dict = try? JSONSerialization.jsonObject(with: info!, options: JSONSerialization.ReadingOptions.mutableContainers) as? [String : Any] {
    let stitchingData:[String: Any] = ["stitchingInfo": dict]
    uploadStitchingInfo = toJSONString(stitching: stitchingData.toJsonString() ?? "")
}

func toJSONString(stitching: Dictionary<String, Any>) -> String {
    guard let data = try? JSONSerialization.data(withJSONObject: stitching,
                                                 options: []) else {
        return ""
    }
    guard let str = String(data: data, encoding: .utf8) else {
        return ""
    }
    let encoder = JSONEncoder()
    encoder.outputFormatting = .prettyPrinted
    guard let data = try? encoder.encode(str) else{ return "" }
    guard let jsonStr = String(data: data, encoding: .utf8) else{ return "" }
    return jsonStr
}

In Video Stitching Mode

Similarly, in video stitching mode, the following logic also needs to be added to the same method: func endTakePhotoCallback(_ stitchingPath: String, _ stitching: String).

/**
 * in function endTakePhotoCallback definition
 * @param stitchingInfo String stitching info
 * @param stitchingInfoVersion 0(classic stitching mode)、1(video stitching mode)、3(ultimate stitching mode)； 
 *        bestPov best stitching position (only classic stitching mode has this information)
 *        resize Image compression ratio (only classic stitching mode has this information)
 *        pair Stitching information of two pictures (only classic stitching mode has this information)
 * @param stitchingPath String stitching local path
 * @param stitchingVersionInfo Int 0(classic stitching mode)、1(video stitching mode)； 
 * */
//overall StitchingInfo is a string, so escape symbols need to be added
let uploadStitchingInfo = "{\\\"stitchingInfo\\\":\(stitchingInfo)}"

In Ultimate Stitching Mode

Similarly, in video stitching mode, the following logic also needs to be added to the same method: func endTakePhotoCallback(_ stitchingPath: String, _ stitching: String).

/**
 * in function endTakePhotoCallback definition
 * @param stitchingInfo String stitching info
 * @param stitchingInfoVersion 0(classic stitching mode)、1(video stitching mode)、3(ultimate stitching mode)； 
 *        bestPov best stitching position (only classic stitching mode has this information)
 *        resize Image compression ratio (only classic stitching mode has this information)
 *        pair Stitching information of two pictures (only classic stitching mode has this information)
 * @param stitchingPath String stitching local path
 * @param stitchingVersionInfo Int 0(classic stitching mode)、1(video stitching mode)； 
 * */
//overall StitchingInfo is a string, so escape symbols need to be added
let uploadStitchingInfo = "{\\\"stitchingInfo\\\":\(stitchingInfo)}"

Unify Logic for Two Modes

If you want to add logic in the method func endTakePhotoCallback(_ stitchingPath: String, _ stitching: String) to handle both modes, here is the complete code:

func endTakePhotoCallback(_ stitchingPath: String, _ stitching: String) {
    let stitchingJSONObj = toDictFromJSONString(jsonString:stitching)
    let stitchingCode:Int = stitchingJSONObj["stitchingInfoVersion"] as! Int
    if (stitchingCode == 0) {
        // classic stitching mode
        let stitchingInfo = "{\\\"stitchingInfo\\\":\(toJSONString(stitching:stitching))}"
        print(stitchingInfo)
    }
    if (stitchingCode == 1) {
        // video stitching mode
        let stitchingInfo = "{\\\"stitchingInfo\\\":\(stitching)}"
        print(stitchingInfo)
    }
    if (stitchingCode == 3) {
        // ultimate stitching mode
        let stitchingInfo = "{\\\"stitchingInfo\\\":\(stitching)}"
        print(stitchingInfo)
    }
    
    // next: needed for OpenAPI: Create Stitching Picture Task param stitching_info https://retail-doc.clobotics.com/en/open-api/api/ir-api#create-stitching-picture-task

    // stitchingInfo taskIds stitchingCode
}

Changelog

For all releases, please visit the Releases.

Category	Item	Version
		< 3.1.11	3.1.11
Config	isUseLargeAngleModel Scope: New apply	Only when userSelectedStitchingMode = 2 && maskStyle = 2 take effect	Also applies when userSelectedStitchingMode = 3
		< 3.1.10	3.1.10
Config	userSelectedStitchingMode Scope: New option	Available: 1 | 2	Available: 1 | 2 | 3 Enable ultimate stitching mode when taking picture
	maxRecordingSecs Scope: New	-	The maximum seconds for recording Note: userSelectedStitchingMode = 3 is required to take effect.
	allowLargeAngle Scope: New	-	Support setting of large angle when detection (0~90°)
	minSkuSizeRatio Scope: New	-	Support setting of min SKU size ratio when detection(0.0~1.0)
		< 3.1.9	3.1.9
Compatibility	Scope: Objective-C support	-	Support Objective-C intergration
		< 3.1.7	3.1.7
Callbacks	cancelTakePhotoCallback Scope: New	-	func cancelTakePhotoCallback(_ imagesPath: [String])
		< 3.1.6	3.1.6
Config	language Scope: New	-	Support setting the display language of the camera UI
		< 3.1.5	3.1.5
Config	maskStyle Scope: New option	Available: 3	Available: 3 | 4 Enable different styles overlay when taking picture, 4 means product price-tag style Note: product price-tag style equals 4 needs userSelectedStitchingMode = 2 to take effect.
		< 3.1.4	3.1.4
Config	enableGEOLocator Scope: New	-	Enable embedding longitude and latitude coordinates into the EXIF data of photo or not
		3.0.0	3.1.0
Callbacks	takePhotoCallback -> takeSinglePhotoCallback Scope: Function name、Parameters	func takePhotoCallback(_ imagePath: String, _ imageIndex: Int, _ imageId: String, _ pair: String, _ stitchingVersionInfo: Int)	func takeSinglePhotoCallback(_ imagePath: String, _ imageIndex: Int, _ imageId: String, _ pair: String)
	endTakePhotoCallback Scope: Parameters	func endTakePhotoCallback(_ stitchingPath: String, _ stitching: String, _ stitchingVersionInfo: Int)	func endTakePhotoCallback(_ stitchingPath: String, _ stitching: String)
Config	maskStyle Scope: New	-	Available: 3 Enable different styles overlay when taking picture, 3 means price-tag style Note: price-tag style equals 3 needs userSelectedStitchingMode = 2 to take effect.