Camera SDK for Android
Integrate the Camera SDK
IMPORTANT During Android intergration, Google Play Policy requires that the compressed APK that users download be no more than 100MB. If your app's compressed download size is larger than 100 MB, you should instead upload your app using Android App Bundles which allows for up to a 200 MB compressed download size.
1. Download SDK
SDK: Android_SDK_v3.1.11Java Latest
SDK Use Demo: Android SDK Demo Source Code | Installation Package Latest
Release Notes: Changelog New
All Versions: Releases
2. Put SDK aar to project
Unarchive the SDK archive file and place clobotics-stitching-camera-\*.\*.\*.aar and clobotics-image-quality-evaluator-\*.\*.\*.aar in the /app/libs folder of your Android application project.
3. Add dependencies
Add the SDK library and other library dependencies in the dependencies section of the build.gradle (Module:app /app/build.gradle) file in the project, just like:
implementation files('libs/clobotics-stitching-camera-*.*.*.aar')
implementation files('libs/clobotics-image-quality-evaluator-*.*.*.aar')
implementation 'com.google.code.gson:gson:2.8.6'
implementation 'com.github.bumptech.glide:glide:4.12.0'4. Permissions Declaration
Declare the following permissions in AndroidManifest.xml file.
<!-- Network Permission -->
<uses-permission android:name="android.permission.INTERNET" />
<!-- Read File Permission -->
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<!-- write File Permission -->
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<!-- Camera Permission -->
<uses-permission android:name="android.permission.CAMERA" />Empowering your app to capture stitching photos with Camera SDK, it is essential to grant the aforementioned four permissions to ensure the proper functioning of the Camera SDK.
5. Release Build Adding ProGuard Rules
If you enable minifyEnabled true in app/build.gradle, add the following rules to your proguard-rules.pro file (to fix common JNI issues, especially UnsatisfiedLinkError caused by JNI_OnLoad failing):
# Keep all classes and classes with native methods in Clobotics Camera SDK's package
-keep class com.clobotics.retail.stitch.R$* { *; }
-keep class com.clobotics.retail.stitch.**{*;}
-keep class com.clobotics.retail.stitch.CloboticsCamera{*;}
-keep class com.clobotics.retail.stitch.StitchingCameraCallback {*;}
-keep class com.clobotics.cvml.** { *; }
-keepclasseswithmembernames class com.clobotics.cvml.$* {
native <methods>;
}
-keep class com.clobotics.retail.zhiwei.** { *; }
-keepclasseswithmembernames class com.clobotics.retail.zhiwei.$* {
native <methods>;
}Refer SDK Use Demo Project
To learn how to interact with the Camera SDK, please unarchive the SDK Use Demo and review in details.
The following sections provide a brief introduction to the core components.
1. Initialize Callback for Camera SDK and Register
//Temporarily store image information, users can define it by themselves, here is just a sample code
List<Object[]> imagesCached = new ArrayList<>();
CloboticsCamera.getInstance().setStitchingCameraCallback(
new StitchingCameraCallback() {
/**
* The callback when a single photo is taken
*
* @param imagePath image save path
* @param imageIndex image number
* @param imageId image unique encoding UUID
* @param pair The information associated with the previous puzzle
* (only classic stitching mode will assign this param, in video stitching mode, would be empty).
* useful for stitchingInfo
* */
@Override
public void takeSinglePhotoCallback(String imagePath, int imageIndex, String imageId, String pair) {
// sample code: Images Cache Manager
imagesCached.add(new Object[]{imagePath, imageIndex, imageId, pair});
// Sort the list of imagesCached by imageIndex in ascending order
Comparator<Object[]> comparator = Comparator.comparingInt(arr -> (int) arr[1]);
Collections.sort(imagesCached, comparator);
}
/**
* The callback when end taking the photo(s) task
*
* @param stitchingPath stitching local path
* @param stitchingInfo stitching info
* stitchingInfoVersion 0 (classic stitching mode), 1 (video stitching mode)
* info puzzle information (video stitching mode will have this information)
* BestPov latest stitching position (only class stitching mode has this information)
* resize puzzle compression ratio (only the class stitching mode has this information)
* */
@Override
public void endTakePhotoCallback(String stitchingPath, JSONObject stitchingInfo) {
//stitch return has the following two situations
//Situation 1: imagesCached.size() == 1,
// so there is no need to initiate a stitch task request to the server, and the 'task_id'
// returned by the single image is used to obtain the recognition result
//Situation 2: imagesCached.size() > 1,
// it is need to initiate stitching task request to the server
// refer to https://retail-doc.clobotics.com/en/open-api/api/ir-api.html#create-stitching-task
int stitchingCode = stitchingInfo.has("stitchingInfoVersion") ? stitchingInfo.getInt("stitchingInfoVersion") : 0;
if (stitchingCode == 0) {
// Please refer to `StitchingInfo management` -> `classic stitching mode` method to generate
}
if (stitchingCode == 1) {
// Please refer to `StitchingInfo management` -> `video stitching mode` method to generate
}
if (stitchingCode == 3) {
// Please refer to `StitchingInfo management` -> `ultimate stitching mode` method to generate
}
}
/**
* The callback when user click CANCEL_BUTTON or navigator back
*
* @param imagesPath images path need to remove
* */
@Override
public void cancelTakePhotoCallback(String imagesPath) {
/** your code here */
}
}
);In the StitchingCameraCallback interface, there are several important callbacks that need attention:
delPhotoCallback: It is invoked when the user cancels a photo during shooting. It's crucial to handle this callback logic, especially in stitching mode, to prevent incorrectStitchingInfo.logCallback: Logs output from the Camera SDK.initFailCallback: It is triggered if the initialization of the Camera SDK fails. It's essential to implement logic to retry the initialization process in response to this callback.
/**
* The callback when photo(s) need to delete
* @param imagesPath image path list
* */
@Override
public void delPhotoCallback(String imagesPath) {
/** your code trys to remove image(s) from imagesCached */
}
/**
* Log callback (keypoints of task, would log)
*
* @param tag keypoint, some examples list as below:
* SupportPreViewSize Resolutions and mem info supported by the device
* isDeviceSupportVideoStitching is device support video stitching mode
* Button_ISO: Camera1 ISO exposure setting
* Button_Torch:Camera1 flashlight setting
* Button_ISO2:Camera2 ISO exposure setting
* Button_Torch2:Camera2 flashlight setting
* Button_Take: Take Photo Button click
* Button_Cancel:Cancel PhotoButton click
* Button_Finish:End Take Photo Button click
* Button_Rollback:Rollback Button Click
* @param message
*/
@Override
public void logCallback(String tag, String message) {
/** your code here */
}
/**
* failed in SDK initialization
* @param message Reason for failure
* */
@Override
public void initFailCallback(String message) {
/** your code here */
}2. Initialize the CameraConfig and Launch Camera Instance
For details on setting CameraConfig, please refer to: Setup Camera Config. Then, use the following code to launch the camera instance.
/**
*
* @param activity
* @param jsonString CameraConfig class for json string,
* Specific attributes list in part: Parameters introduction for config
* @param requestCode (for example:0x001)
*/
CloboticsCamera.getInstance().startCameraStitching(Activity activity, String jsonString, int requestCode);3. Handle Exceptions
Exceptions are typically triggered by system back events or abnormal exits.
@Override
public void onActivityResult(int requestCode, int resultCode, Intent data) {
super.onActivityResult(requestCode, resultCode, data);
// check the coresponding request code
if (resultCode != RESULT_OK) {
// Abnormal exist, clear all data of Task
// TODO DeleteTask data
imagesCached.clear();
}
}4. StitchingInfo Management
The crucial part of capturing stitching images is properly managing the StitchingInfo. If not managed correctly, when calling the Create Stitching Picture Task in the OpenAPI system, the task may fail due to incorrect information. Please read StitchingInfo Management carefully for detailed information.
Setup Camera Config
1. Two Methods to Setup
- For all attributes of this class, please refer Attributes of CameraConfig class.
When invoking the method CloboticsCamera.getInstance().startCameraStitching(Activity activity, String configJsonString, int requestCode), the second parameter is a raw json string converted from a specified JSONObject. There are two ways to set up the camara config when using this method:
Option 1: Use
com.clobotics.retail.stitch.utils.CameraConfigto set up the configuration, convert it to aJSONObject, and then convert it to a string.Option 2: Use a raw json string with the proper JSON format directly.
/**
* Use simplest config, other params in default.
*
*/
CloboticsCamera.getInstance().startCameraStitching(..., configJsonString, ...);2. Stitching Modes
Classic Stitching Mode
- Option 1: Use
CameraConfigclass
import com.clobotics.retail.stitch.utils.CameraConfig;
import com.google.gson.Gson;
CameraConfig config = new CameraConfig();
// 1 video stitching, 2 classic stitching 3 ultimate stitching
config.setUserSelectedStitchingMode(2);
config.setMaskStyle(2); // can be omitted
config.setStitchingMinCount(2);
config.setStitchingMaxCount(4);
String configJsonString = new Gson().toJson(config);- Option 2: Use raw string
String configJsonString = "{\"userSelectedStitchingMode\": 2, \"maskStyle\": 2, \"stitchingMinCount\": 2, \"stitchingMaxCount\": 4}";And then invoke the startCameraStitching() method.
Notes:
- For the
stitchingMinCountandstitchingMaxCountparameters, set them to the appropriate values based on your requirements.
Video Stitching Mode
- Option 1: Use
CameraConfigclass
import com.clobotics.retail.stitch.utils.CameraConfig;
import com.google.gson.Gson;
CameraConfig config = new CameraConfig();
// 1 video stitching, 2 classic stitching 3 ultimate stitching
config.setUserSelectedStitchingMode(1);
config.setStitchingMinCount(2);
config.setStitchingMaxCount(4);
String configJsonString = new Gson().toJson(config);- Option 2: Use raw json string
String configJsonString = "{\"userSelectedStitchingMode\": 1, \"stitchingMinCount\": 2, \"stitchingMaxCount\": 4}";And then invoke the startCameraStitching() method.
Notes:
- For the
stitchingMinCountandstitchingMaxCountparameters, set them to the appropriate values based on your requirements.
Ultimate Stitching Mode
- Option 1: Use
CameraConfigclass
import com.clobotics.retail.stitch.utils.CameraConfig;
import com.google.gson.Gson;
CameraConfig config = new CameraConfig();
// 1 video stitching, 2 classic stitching 3 ultimate stitching
config.setUserSelectedStitchingMode(3);
config.setStitchingMinCount(2);
config.setMaxRecordingSecs(45);
String configJsonString = new Gson().toJson(config);- Option 2: Use raw json string
String configJsonString = "{\"userSelectedStitchingMode\": 3, \"stitchingMinCount\": 2, \"maxRecordingSecs\": 45}";And then invoke the startCameraStitching() method.
Notes:
- For the
stitchingMinCountandmaxRecordingSecsparameters, set them to the appropriate values based on your requirements.
3. Single Capture Modes
Classic Single Capture Mode
In this mode, users can still capture multiple images simultaneously without exiting the camera. Unlike stitching mode, although multiple images are captured, the Camera SDK will not provide StitchingInfo.
- Option 1: Use
CameraConfigclass
import com.clobotics.retail.stitch.utils.CameraConfig;
import com.google.gson.Gson;
CameraConfig config = new CameraConfig();
config.setUserSelectedStitchingMode(2);
config.setMaskStyle(1);- Option 2: Use raw JSON string
String configJsonString = "{\"userSelectedStitchingMode\": 2, \"maskStyle\": 1}";And then invoke the startCameraStitching()
Notes:
- For the
stitchingMinCountandstitchingMaxCountparameters, set them to the appropriate values based on your requirements.
Pure Price-Tag Capture Mode
This mode introduces a guide mask for scenarios involving the capture of price tags, which is a particular scenario of Classic Stitching Mode. In this mode, users can capture multiple single images with the mask displayed simultaneously without exiting the camera. The significant difference in this mode is that, although there are multiple images, Camera SDK won't provide StitchingInfo, and you also don't need to manage the StitchingInfo.
- Option 1: Use
CameraConfigclass
import com.clobotics.retail.stitch.utils.CameraConfig;
import com.google.gson.Gson;
CameraConfig config = new CameraConfig();
config.setUserSelectedStitchingMode(2);
config.setStitchingMinCount(1);
config.setStitchingMaxCount(1);
config.setMaskStyle(3);
String configJsonString = new Gson().toJson(config);- Option 2: Use raw json string
String configJsonString = "{\"userSelectedStitchingMode\": 2, \"maskStyle\": 3, \"stitchingMinCount\": 1, \"stitchingMaxCount\": 1}";And then invoke the startCameraStitching() method.
Product Price-Tag Capture Mode
This mode introduces a guide mask for scenarios involving the capture of price tag and product, which is a particular scenario of Classic Stitching Mode. In this mode, users can capture multiple single images with the mask displayed simultaneously without exiting the camera. The significant difference in this mode is that, although there are multiple images, Camera SDK won't provide StitchingInfo, and you also don't need to manage the StitchingInfo.
- Option 1: Use
CameraConfigclass
import com.clobotics.retail.stitch.utils.CameraConfig;
import com.google.gson.Gson;
CameraConfig config = new CameraConfig();
config.setUserSelectedStitchingMode(2);
config.setStitchingMinCount(1);
config.setStitchingMaxCount(1);
config.setMaskStyle(4);
String configJsonString = new Gson().toJson(config);- Option 2: Use raw json string
String configJsonString = "{\"userSelectedStitchingMode\": 2, \"maskStyle\": 4, \"stitchingMinCount\": 1, \"stitchingMaxCount\": 1}";And then invoke the startCameraStitching() method.
Multi-language Support
Supported languages include:
| Language | Parameter Value |
|---|---|
| English | Locale.ENGLISH |
| Simplified Chinese | Locale.CHINESE |
| Traditional Chinese | new Locale("zh", "Hant") |
| Thai | new Locale("th") |
| Myanmar | new Locale("my") |
| French | new Locale("fr") |
| Italian | new Locale("it") |
| Portuguese | new Locale("pt") |
| Spanish | new Locale("es") |
- Option 1: Use the
CameraConfigclass
import java.util.Locale;
CameraConfig config = new CameraConfig();
config.setLanguage(Locale.ENGLISH.toString());
String configJsonString = new Gson().toJson(config);- Option 2: Use a raw JSON string
String lang = Locale.ENGLISH.toString();
String configJsonString = String.format("{\"language\": \"%s\"}", lang);Attributes of CameraConfig class
| Parts | Parameter | Description | Type | Default | Version |
|---|---|---|---|---|---|
| Multi-language Support | language | Set the display language. Supported languages | string | Follow system | 3.1.6 |
| Stitching | userSelectedStitchingMode | capture mode in default when starts, 1 means video stitching, 2 means classic capture mode(aligned with maskStyle assigned can specify stiching capture, single image capture or price-tag capture mode), 3 means ultimate stitching Note: when 1 assigned,needs hardware support, then video stitching would take effect | int | 2 | 3.0.0 |
| maskStyle | Enable different styles overlay when taking picture: 1 means no guide frame (normal capture); 3 means pure price-tag guide frame; 4 means product price-tag guide frame Note: When configured as 1, 3, or 4, userSelectedStitchingMode = 2 is required to take effect. | int | 0 | 3.1.5 | |
| isFirstUse | Enable first-time usage guide or not | boolean | false | 3.0.0 | |
| allowRoll | Allow the user to rotate the angle along the Roll direction(0~90°) | float | 45 | 3.0.0 | |
| allowPitch | Allow the user to swing the angle along the Pitch direction(0~90°) | float | 45 | 3.0.0 | |
| stitchingMinCount | The minimum photo count for take a stitching task, 0 means not limit | int | 0 | 3.0.0 | |
| stitchingMaxCount | The maximum photo count for take a stitching task, 0 means not limit | int | 0 | 3.0.0 | |
| maxRecordingSecs | The maximum seconds for recording Note: userSelectedStitchingMode = 3 is required to take effect. | int | 0 | 3.1.10 | |
| enableGEOLocator | Enable embedding longitude and latitude coordinates into the EXIF data of photo or not | boolean | false | 3.1.4 | |
| Image Quality Check | isUseLargeAngleModel | Enable large angle detection Note: only when classic stitching (userSelectedStitchingMode = 2 && maskStyle = 2) or ultimate stitching (userSelectedStitchingMode = 3), isUseLargeAngleModel would take effect. Single Capture Modes won't take effect. | boolean | false | 3.1.11 |
| allowLargeAngle | The maximum large angle when detection enabled(0~90°) Note: isUseLargeAngleModel=true is required to take effect. | float | 20.0 | 3.1.10 | |
| isUseSkuSizeModel | Enable check SKU-part's proportion of single picture(if enabled 30% in default) Note: only when userSelectedStitchingMode = 2 && maskStyle = 2, isUseSkuSizeModel would take effect | boolean | false | 3.0.0 | |
| minSkuSizeRatio | The minimum of SKU size ratio required(0.0~1.0) Note: isUseSkuSizeModel=true is required to take effect. | float | 0.3 | 3.1.10 |
StitchingInfo management
In Classic Stitching Mode
The following logic must be added to the public void endTakePhotoCallback(String stitchingPath, JSONObject stitchingInfo) method, as shown in the SDK Use Demo project.
/**
* 1.global variable Array imageCached definition:
* List<Object[]> imagesCached = new ArrayList<>();
* 2.add image to Array imageCached in function takePhotoCallback
* imagesCached.add(new Object[]{imagePath, imageIndex, ...});
* 3.only when imagesCached.size() > 1, stitchingInfo is needed for stitching task for recognition
* */
JSONArray pairArray = new JSONArray();
// Object[] image = [imagePath, imageIndex, imageId, pair, taskId]
// image's pair info starts from image(i = 1), because image(i=0) pair info is empty
for (int i = 1; i < imagesCached.size(); i++) {
pairArray.put(new JSONObject(imagesCached.get(i)[3].toString()));
}
stitchingInfo.put("pair", pairArray);
JSONObject jsonObject = new JSONObject();
jsonObject.put("stitchingInfo", stitchingInfo);
String uploadStitchingInfo = jsonObject.toString();In Video Stitching Mode
Similarly, in video stitching mode, the following logic also needs to be added to the same method: public void endTakePhotoCallback(String stitchingPath, JSONObject stitchingInfo).
JSONObject jsonObject = new JSONObject();
jsonObject.put("stitchingInfo", new JSONObject(stitchingInfo.getString("info")));
String uploadStitchingInfo = jsonObject.toString();In Ultimate Stitching Mode
Similarly, in ultimate stitching mode, the following logic also needs to be added to the same method: public void endTakePhotoCallback(String stitchingPath, JSONObject stitchingInfo).
JSONObject jsonObject = new JSONObject();
jsonObject.put("stitchingInfo", new JSONObject(stitchingInfo.getString("info")));
String uploadStitchingInfo = jsonObject.toString();Unify Logic for Stitching Modes
If you want to add logic in the method public void endTakePhotoCallback(String stitchingPath, JSONObject stitchingInfo) to handle both modes, here is the complete code:
@Override
public void endTakePhotoCallback(String stitchingPath, JSONObject stitchingInfo) {
super.endTakePhotoCallback(stitchingPath, stitchingInfo);
print("end:"+stitchingInfo.toString()+":"+groupTaskId+":"+stitchingPath);
try {
JSONObject jsonObject = new JSONObject();
int stitchingInfoVersion = stitchingInfo.has("stitchingInfoVersion") ? stitchingInfo.getInt("stitchingInfoVersion") : 0;
// video stitching mode or ultimate stitching mode
if (stitchingInfoVersion == 1 || stitchingInfoVersion == 3) {
jsonObject.put("stitchingInfo", new JSONObject(stitchingInfo.getString("info")));
} else { // classic stitching mode
JSONArray pairArray = new JSONArray();
// Object[] image = [imagePath, imageIndex, imageId, pair, taskId]
// image's pair info starts from image(i = 1), because image(i=0) pair info is empty
for (int i = 1; i < images.size(); i++) {
Object[] image = images.get(i);
pairArray.put(new JSONObject(image[3].toString()));
}
stitchingInfo.put("pair", pairArray);
jsonObject.put("stitchingInfo", stitchingInfo);
}
// needed for OpenAPI: Create Stitching Picture Task param stitching_info https://retail-doc.clobotics.com/en/open-api/api/ir-api#create-stitching-picture-task
String uploadStitchingInfo = jsonObject.toString();
print(uploadStitchingInfo);
} catch (JSONException e) {
e.printStackTrace();
}
}Notes:
uploadStitchingInforepresents thestitching_infoparameter required by the Create Stitching Picture Task API.stitchingInfoVersioncorresponds to thestitching_info_versionparameter needed for the Create Stitching Picture Task API.
Changelog
For all releases, please visit the Releases.
| Category | Item | Version | |
|---|---|---|---|
| < 3.1.11 | 3.1.11 | ||
| Config | isUseLargeAngleModel Scope: New apply | Only when userSelectedStitchingMode = 2 && maskStyle = 2 take effect | Also applies when userSelectedStitchingMode = 3 |
| < 3.1.10 | 3.1.10 | ||
| Config | userSelectedStitchingMode Scope: New option | Available: 1 | 2 | Available: 1 | 2 | 3 Enable ultimate stitching mode when taking picture |
| maxRecordingSecs Scope: New | - | The maximum seconds for recording Note: userSelectedStitchingMode = 3 is required to take effect. | |
| allowLargeAngle Scope: New | - | Support setting of large angle when detection (0~90°) | |
| minSkuSizeRatio Scope: New | - | Support setting of min SKU size ratio when detection(0.0~1.0) | |
| < 3.1.8 | 3.1.8 | ||
| Dependencies | Scope: Removed | Dependencies declaration MUST include implementation 'fr.avianey.com.viewpagerindicator:library:2.4.1@aar' | Removed |
| Compatibility | Scope: Alignment | - | Support 16 KB memory page sizes |
| < 3.1.7 | 3.1.7 | ||
| Callbacks | cancelTakePhotoCallback Scope: New | - | public void cancelTakePhotoCallback(String imagesPath) |
| < 3.1.6 | 3.1.6 | ||
| Config | language Scope: New | - | Support setting the display language of the camera UI |
| < 3.1.5 | 3.1.5 | ||
| Config | maskStyle Scope: New option | Available: 3 | Available: 3 | 4 Enable different styles overlay when taking picture, 4 means product price-tag style Note: product price-tag style equals 4 needs userSelectedStitchingMode = 2 to take effect. |
| < 3.1.4 | 3.1.4 | ||
| Config | enableGEOLocator Scope: New | - | Enable embedding longitude and latitude coordinates into the EXIF data of photo or not |
| < 3.1.2 | 3.1.2 | ||
| Class | CameraConfig Scope: route change | import com.clobotics.retail.stitch.utils.CameraConfig; | import com.clobotics.retail.stitch.CameraConfig; |
| 3.0.0 | 3.1.0 | ||
| Callbacks | takePhotoCallback -> takeSinglePhotoCallback Scope: Function name、Parameters | public void takePhotoCallback(String imagePath, int imageIndex, String imageId, String pair, int taskId) | public void takeSinglePhotoCallback(String imagePath, int imageIndex, String imageId, String pair) |
| endTakePhotoCallback Scope: Parameters | public void endTakePhotoCallback(JSONObject stitchingInfo, int taskId, String groupTaskId, String stitchingPath) | public void endTakePhotoCallback(String stitchingPath, JSONObject stitchingInfo) | |
| startTakePhotoCallback Scope: Delete | public int startTakePhotoCallback(String groupTaskId, int planId, int sceneId) | - | |
| Config | maskStyle Scope: New | - | Enable different styles overlay when taking picture, 3 means pure price-tag style Note: pure price-tag style equals 3 needs userSelectedStitchingMode = 2 to take effect. |