Skip to content

Camera SDK for Android

Integrate the Camera SDK

IMPORTANT During Android intergration, Google Play Policy requires that the compressed APK that users download be no more than 100MB. If your app's compressed download size is larger than 100 MB, you should instead upload your app using Android App Bundles which allows for up to a 200 MB compressed download size.

1. Download SDK

SDK: Android_SDK_v3.1.11Java Latest

SDK Use Demo: Android SDK Demo Source Code Installation Package  Latest

Release Notes: Changelog New

All Versions: Releases

2. Put SDK aar to project

Unarchive the SDK archive file and place clobotics-stitching-camera-\*.\*.\*.aar and clobotics-image-quality-evaluator-\*.\*.\*.aar in the /app/libs folder of your Android application project.

3. Add dependencies

Add the SDK library and other library dependencies in the dependencies section of the build.gradle (Module:app /app/build.gradle) file in the project, just like:

groovy
implementation files('libs/clobotics-stitching-camera-*.*.*.aar')  
implementation files('libs/clobotics-image-quality-evaluator-*.*.*.aar')  

implementation 'com.google.code.gson:gson:2.8.6'  
implementation 'com.github.bumptech.glide:glide:4.12.0'

4. Permissions Declaration

Declare the following permissions in AndroidManifest.xml file.

xml
<!-- Network Permission -->
<uses-permission android:name="android.permission.INTERNET" />
<!-- Read File Permission -->
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" /> 
<!-- write File Permission -->
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<!-- Camera Permission -->
<uses-permission android:name="android.permission.CAMERA" />

Empowering your app to capture stitching photos with Camera SDK, it is essential to grant the aforementioned four permissions to ensure the proper functioning of the Camera SDK.

5. Release Build Adding ProGuard Rules

If you enable minifyEnabled true in app/build.gradle, add the following rules to your proguard-rules.pro file (to fix common JNI issues, especially UnsatisfiedLinkError caused by JNI_OnLoad failing):

proguard
# Keep all classes and classes with native methods in Clobotics Camera SDK's package
-keep class com.clobotics.retail.stitch.R$* { *; }
-keep class com.clobotics.retail.stitch.**{*;}
-keep class com.clobotics.retail.stitch.CloboticsCamera{*;}
-keep class com.clobotics.retail.stitch.StitchingCameraCallback {*;}

-keep class com.clobotics.cvml.** { *; }
-keepclasseswithmembernames class com.clobotics.cvml.$* {
    native <methods>;
}

-keep class com.clobotics.retail.zhiwei.** { *; }
-keepclasseswithmembernames class com.clobotics.retail.zhiwei.$* {
    native <methods>;
}

Refer SDK Use Demo Project

To learn how to interact with the Camera SDK, please unarchive the SDK Use Demo and review in details.

The following sections provide a brief introduction to the core components.

1. Initialize Callback for Camera SDK and Register

java

//Temporarily store image information, users can define it by themselves, here is just a sample code
List<Object[]> imagesCached = new ArrayList<>(); 

CloboticsCamera.getInstance().setStitchingCameraCallback(
    new StitchingCameraCallback() {
         /**
          * The callback when a single photo is taken
          *
          * @param imagePath image save path
          * @param imageIndex image number
          * @param imageId image unique encoding UUID
          * @param pair The information associated with the previous puzzle 
          * (only classic stitching mode will assign this param, in video stitching mode, would be empty).
          *  useful for stitchingInfo 
          * */
         @Override
         public void takeSinglePhotoCallback(String imagePath, int imageIndex, String imageId, String pair) {
            // sample code: Images Cache Manager
            imagesCached.add(new Object[]{imagePath, imageIndex, imageId, pair}); 

            // Sort the list of imagesCached by imageIndex in ascending order
            Comparator<Object[]> comparator = Comparator.comparingInt(arr -> (int) arr[1]);
            Collections.sort(imagesCached, comparator);
         }

         /**
           * The callback when end taking the photo(s) task
           * 
           * @param stitchingPath stitching local path
           * @param stitchingInfo stitching info
           *    stitchingInfoVersion 0 (classic stitching mode), 1 (video stitching mode)
           *    info puzzle information (video stitching mode will have this information)
           *    BestPov latest stitching position (only class stitching mode has this information)
           *    resize puzzle compression ratio (only the class stitching mode has this information)
           * */
         @Override
         public void endTakePhotoCallback(String stitchingPath, JSONObject stitchingInfo) {
            //stitch return has the following two situations
            //Situation 1: imagesCached.size() == 1, 
            // so there is no need to initiate a stitch task request to the server, and the 'task_id' 
            // returned by the single image is used to obtain the recognition result
            //Situation 2: imagesCached.size() > 1, 
            // it is need to initiate stitching task request to the server
            // refer to https://retail-doc.clobotics.com/en/open-api/api/ir-api.html#create-stitching-task
            int stitchingCode = stitchingInfo.has("stitchingInfoVersion") ? stitchingInfo.getInt("stitchingInfoVersion") : 0;
            if (stitchingCode == 0) {
                // Please refer to `StitchingInfo management` -> `classic stitching mode` method to generate
            }
            if (stitchingCode == 1) {
                // Please refer to `StitchingInfo management` -> `video stitching mode` method to generate
            }
            if (stitchingCode == 3) {
                // Please refer to `StitchingInfo management` -> `ultimate stitching mode` method to generate
            }
         }
         /**
          * The callback when user click CANCEL_BUTTON or navigator back
          * 
          * @param imagesPath images path need to remove
          * */
        @Override
        public void cancelTakePhotoCallback(String imagesPath) {
            /** your code here */
        }
    }
);

In the StitchingCameraCallback interface, there are several important callbacks that need attention:

  • delPhotoCallback: It is invoked when the user cancels a photo during shooting. It's crucial to handle this callback logic, especially in stitching mode, to prevent incorrect StitchingInfo.

  • logCallback: Logs output from the Camera SDK.

  • initFailCallback: It is triggered if the initialization of the Camera SDK fails. It's essential to implement logic to retry the initialization process in response to this callback.

java
    /**
      * The callback when photo(s) need to delete
      * @param imagesPath image path list
      * */
     @Override
     public void delPhotoCallback(String imagesPath) {
         /** your code trys to remove image(s) from imagesCached */
     }

     /**
      * Log callback (keypoints of task, would log)
      * 
      * @param tag  keypoint, some examples list as below: 
      *     SupportPreViewSize Resolutions and mem info supported by the device
      *     isDeviceSupportVideoStitching is device support video stitching mode
      *     Button_ISO: Camera1 ISO exposure setting
      *     Button_Torch:Camera1 flashlight setting
      *     Button_ISO2:Camera2 ISO exposure setting
      *     Button_Torch2:Camera2 flashlight setting
      *     Button_Take: Take Photo Button click
      *     Button_Cancel:Cancel PhotoButton click
      *     Button_Finish:End Take Photo Button click
      *     Button_Rollback:Rollback Button Click
      * @param message
      */
     @Override
     public void logCallback(String tag, String message) {
         /** your code here */
     }

     /**
      * failed in SDK initialization
      * @param message Reason for failure
      * */
     @Override
     public void initFailCallback(String message) {
         /** your code here */
     }

2. Initialize the CameraConfig and Launch Camera Instance

For details on setting CameraConfig, please refer to: Setup Camera Config. Then, use the following code to launch the camera instance.

java
/**
 *
 * @param activity 
 * @param jsonString CameraConfig class for json string,
 *      Specific attributes list in part: Parameters introduction for config
 * @param requestCode (for example:0x001)
 */
CloboticsCamera.getInstance().startCameraStitching(Activity activity, String jsonString, int requestCode);

3. Handle Exceptions

Exceptions are typically triggered by system back events or abnormal exits.

java
@Override
public void onActivityResult(int requestCode, int resultCode, Intent data) {
   super.onActivityResult(requestCode, resultCode, data);
   // check the coresponding request code
   if (resultCode != RESULT_OK) { 
      // Abnormal exist, clear all data of Task
      // TODO DeleteTask data
     imagesCached.clear();
   }
}

4. StitchingInfo Management

The crucial part of capturing stitching images is properly managing the StitchingInfo. If not managed correctly, when calling the Create Stitching Picture Task in the OpenAPI system, the task may fail due to incorrect information. Please read StitchingInfo Management carefully for detailed information.

Setup Camera Config

1. Two Methods to Setup

When invoking the method CloboticsCamera.getInstance().startCameraStitching(Activity activity, String configJsonString, int requestCode), the second parameter is a raw json string converted from a specified JSONObject. There are two ways to set up the camara config when using this method:

  • Option 1: Use com.clobotics.retail.stitch.utils.CameraConfig to set up the configuration, convert it to a JSONObject, and then convert it to a string.

  • Option 2: Use a raw json string with the proper JSON format directly.

java
/**
 * Use simplest config, other params in default.
 * 
 */
CloboticsCamera.getInstance().startCameraStitching(..., configJsonString, ...);

2. Stitching Modes

Classic Stitching Mode

  • Option 1: Use CameraConfig class
java
import com.clobotics.retail.stitch.utils.CameraConfig;
import com.google.gson.Gson;

CameraConfig config = new CameraConfig();
// 1 video stitching, 2 classic stitching 3 ultimate stitching
config.setUserSelectedStitchingMode(2);     
config.setMaskStyle(2);  // can be omitted
config.setStitchingMinCount(2);
config.setStitchingMaxCount(4);

String configJsonString = new Gson().toJson(config);
  • Option 2: Use raw string
java
String configJsonString = "{\"userSelectedStitchingMode\": 2, \"maskStyle\": 2, \"stitchingMinCount\": 2, \"stitchingMaxCount\": 4}";

And then invoke the startCameraStitching() method.

Notes:

  • For the stitchingMinCount and stitchingMaxCount parameters, set them to the appropriate values based on your requirements.

Video Stitching Mode

  • Option 1: Use CameraConfig class
java
import com.clobotics.retail.stitch.utils.CameraConfig;
import com.google.gson.Gson;


CameraConfig config = new CameraConfig();
// 1 video stitching,  2 classic stitching  3 ultimate stitching
config.setUserSelectedStitchingMode(1);     
config.setStitchingMinCount(2);
config.setStitchingMaxCount(4);

String configJsonString = new Gson().toJson(config);
  • Option 2: Use raw json string
java
String configJsonString = "{\"userSelectedStitchingMode\": 1, \"stitchingMinCount\": 2, \"stitchingMaxCount\": 4}";

And then invoke the startCameraStitching() method.

Notes:

  • For the stitchingMinCount and stitchingMaxCount parameters, set them to the appropriate values based on your requirements.

Ultimate Stitching Mode

  • Option 1: Use CameraConfig class
java
import com.clobotics.retail.stitch.utils.CameraConfig;
import com.google.gson.Gson;


CameraConfig config = new CameraConfig();
// 1 video stitching,  2 classic stitching  3 ultimate stitching
config.setUserSelectedStitchingMode(3);     
config.setStitchingMinCount(2);
config.setMaxRecordingSecs(45);

String configJsonString = new Gson().toJson(config);
  • Option 2: Use raw json string
java
String configJsonString = "{\"userSelectedStitchingMode\": 3, \"stitchingMinCount\": 2, \"maxRecordingSecs\": 45}";

And then invoke the startCameraStitching() method.

Notes:

  • For the stitchingMinCount and maxRecordingSecs parameters, set them to the appropriate values based on your requirements.

3. Single Capture Modes

Classic Single Capture Mode

In this mode, users can still capture multiple images simultaneously without exiting the camera. Unlike stitching mode, although multiple images are captured, the Camera SDK will not provide StitchingInfo.

  • Option 1: Use CameraConfig class
java
import com.clobotics.retail.stitch.utils.CameraConfig;
import com.google.gson.Gson;

CameraConfig config = new CameraConfig();
config.setUserSelectedStitchingMode(2);  
config.setMaskStyle(1);
  • Option 2: Use raw JSON string
java
String configJsonString = "{\"userSelectedStitchingMode\": 2, \"maskStyle\": 1}";

And then invoke the startCameraStitching()

Notes:

  • For the stitchingMinCount and stitchingMaxCount parameters, set them to the appropriate values based on your requirements.

Pure Price-Tag Capture Mode

This mode introduces a guide mask for scenarios involving the capture of price tags, which is a particular scenario of Classic Stitching Mode. In this mode, users can capture multiple single images with the mask displayed simultaneously without exiting the camera. The significant difference in this mode is that, although there are multiple images, Camera SDK won't provide StitchingInfo, and you also don't need to manage the StitchingInfo.

  • Option 1: Use CameraConfig class
java
import com.clobotics.retail.stitch.utils.CameraConfig;
import com.google.gson.Gson;

CameraConfig config = new CameraConfig();
config.setUserSelectedStitchingMode(2);     
config.setStitchingMinCount(1);
config.setStitchingMaxCount(1);
config.setMaskStyle(3);

String configJsonString = new Gson().toJson(config);
  • Option 2: Use raw json string
java
String configJsonString = "{\"userSelectedStitchingMode\": 2, \"maskStyle\": 3, \"stitchingMinCount\": 1, \"stitchingMaxCount\": 1}";

And then invoke the startCameraStitching() method.

Product Price-Tag Capture Mode

This mode introduces a guide mask for scenarios involving the capture of price tag and product, which is a particular scenario of Classic Stitching Mode. In this mode, users can capture multiple single images with the mask displayed simultaneously without exiting the camera. The significant difference in this mode is that, although there are multiple images, Camera SDK won't provide StitchingInfo, and you also don't need to manage the StitchingInfo.

  • Option 1: Use CameraConfig class
java
import com.clobotics.retail.stitch.utils.CameraConfig;
import com.google.gson.Gson;

CameraConfig config = new CameraConfig();
config.setUserSelectedStitchingMode(2);     
config.setStitchingMinCount(1);
config.setStitchingMaxCount(1);
config.setMaskStyle(4);

String configJsonString = new Gson().toJson(config);
  • Option 2: Use raw json string
java
String configJsonString = "{\"userSelectedStitchingMode\": 2, \"maskStyle\": 4, \"stitchingMinCount\": 1, \"stitchingMaxCount\": 1}";

And then invoke the startCameraStitching() method.

Multi-language Support

Supported languages include:

LanguageParameter Value
EnglishLocale.ENGLISH
Simplified ChineseLocale.CHINESE
Traditional Chinesenew Locale("zh", "Hant")
Thainew Locale("th")
Myanmarnew Locale("my")
Frenchnew Locale("fr")
Italiannew Locale("it")
Portuguesenew Locale("pt")
Spanishnew Locale("es")
  • Option 1: Use the CameraConfig class
java
import java.util.Locale;

CameraConfig config = new CameraConfig();
config.setLanguage(Locale.ENGLISH.toString());

String configJsonString = new Gson().toJson(config);
  • Option 2: Use a raw JSON string
java
String lang = Locale.ENGLISH.toString();
String configJsonString = String.format("{\"language\": \"%s\"}", lang);

Attributes of CameraConfig class

PartsParameterDescriptionTypeDefaultVersion
Multi-language SupportlanguageSet the display language. Supported languagesstringFollow system3.1.6
StitchinguserSelectedStitchingModecapture mode in default when starts, 1 means video stitching, 2 means classic capture mode(aligned with maskStyle assigned can specify stiching capture, single image capture or price-tag capture mode), 3 means ultimate stitching
Note: when 1 assigned,needs hardware support, then video stitching would take effect
int23.0.0
maskStyleEnable different styles overlay when taking picture: 1 means no guide frame (normal capture); 3 means pure price-tag guide frame; 4 means product price-tag guide frame
Note: When configured as 1, 3, or 4, userSelectedStitchingMode = 2 is required to take effect.
int03.1.5
isFirstUseEnable first-time usage guide or not booleanfalse3.0.0
allowRollAllow the user to rotate the angle along the Roll direction(0~90°)float453.0.0
allowPitchAllow the user to swing the angle along the Pitch direction(0~90°)float453.0.0
stitchingMinCountThe minimum photo count for take a stitching task, 0 means not limitint03.0.0
stitchingMaxCountThe maximum photo count for take a stitching task, 0 means not limitint03.0.0
maxRecordingSecsThe maximum seconds for recording
Note: userSelectedStitchingMode = 3 is required to take effect.
int03.1.10
enableGEOLocatorEnable embedding longitude and latitude coordinates into the EXIF data of photo or notbooleanfalse3.1.4
Image Quality CheckisUseLargeAngleModelEnable large angle detection
Note: only when classic stitching (userSelectedStitchingMode = 2 && maskStyle = 2) or ultimate stitching (userSelectedStitchingMode = 3), isUseLargeAngleModel would take effect. Single Capture Modes won't take effect.
booleanfalse3.1.11
allowLargeAngleThe maximum large angle when detection enabled(0~90°)
Note: isUseLargeAngleModel=true is required to take effect.
float20.03.1.10
isUseSkuSizeModelEnable check SKU-part's proportion of single picture(if enabled 30% in default)
Note: only when userSelectedStitchingMode = 2 && maskStyle = 2, isUseSkuSizeModel would take effect
booleanfalse3.0.0
minSkuSizeRatioThe minimum of SKU size ratio required(0.0~1.0)
Note: isUseSkuSizeModel=true is required to take effect.
float0.33.1.10

StitchingInfo management

In Classic Stitching Mode

The following logic must be added to the public void endTakePhotoCallback(String stitchingPath, JSONObject stitchingInfo) method, as shown in the SDK Use Demo project.

java
/** 
 * 1.global variable Array imageCached definition: 
 *  List<Object[]> imagesCached = new ArrayList<>(); 
 * 2.add image to Array imageCached in function takePhotoCallback
 *  imagesCached.add(new Object[]{imagePath, imageIndex, ...}); 
 * 3.only when imagesCached.size() > 1, stitchingInfo is needed for stitching task for recognition
 * */

JSONArray pairArray = new JSONArray();
// Object[] image = [imagePath, imageIndex, imageId, pair, taskId]
// image's pair info starts from image(i = 1), because image(i=0) pair info is empty
for (int i = 1; i < imagesCached.size(); i++) {
    pairArray.put(new JSONObject(imagesCached.get(i)[3].toString()));
}
stitchingInfo.put("pair", pairArray);

JSONObject jsonObject = new JSONObject();
jsonObject.put("stitchingInfo", stitchingInfo);
String uploadStitchingInfo = jsonObject.toString();

In Video Stitching Mode

Similarly, in video stitching mode, the following logic also needs to be added to the same method: public void endTakePhotoCallback(String stitchingPath, JSONObject stitchingInfo).

java
JSONObject jsonObject = new JSONObject();
jsonObject.put("stitchingInfo", new JSONObject(stitchingInfo.getString("info")));
String uploadStitchingInfo = jsonObject.toString();

In Ultimate Stitching Mode

Similarly, in ultimate stitching mode, the following logic also needs to be added to the same method: public void endTakePhotoCallback(String stitchingPath, JSONObject stitchingInfo).

java
JSONObject jsonObject = new JSONObject();
jsonObject.put("stitchingInfo", new JSONObject(stitchingInfo.getString("info")));
String uploadStitchingInfo = jsonObject.toString();

Unify Logic for Stitching Modes

If you want to add logic in the method public void endTakePhotoCallback(String stitchingPath, JSONObject stitchingInfo) to handle both modes, here is the complete code:

java
@Override
public void endTakePhotoCallback(String stitchingPath, JSONObject stitchingInfo) {
    super.endTakePhotoCallback(stitchingPath, stitchingInfo);
    print("end:"+stitchingInfo.toString()+":"+groupTaskId+":"+stitchingPath);
    try {
        JSONObject jsonObject = new JSONObject();
        int stitchingInfoVersion = stitchingInfo.has("stitchingInfoVersion") ? stitchingInfo.getInt("stitchingInfoVersion") : 0;
        // video stitching mode or ultimate stitching mode
        if (stitchingInfoVersion == 1 || stitchingInfoVersion == 3) { 
            jsonObject.put("stitchingInfo", new JSONObject(stitchingInfo.getString("info")));
        } else { // classic stitching mode
            JSONArray pairArray = new JSONArray();
            // Object[] image = [imagePath, imageIndex, imageId, pair, taskId]
            // image's pair info starts from image(i = 1), because image(i=0) pair info is empty
            for (int i = 1; i < images.size(); i++) {
                Object[] image = images.get(i);
                pairArray.put(new JSONObject(image[3].toString()));
            }
            stitchingInfo.put("pair", pairArray);
            jsonObject.put("stitchingInfo", stitchingInfo);
        }
        // needed for OpenAPI: Create Stitching Picture Task param stitching_info https://retail-doc.clobotics.com/en/open-api/api/ir-api#create-stitching-picture-task
        String uploadStitchingInfo = jsonObject.toString();
        print(uploadStitchingInfo);
    } catch (JSONException e) {
        e.printStackTrace();
    }
}

Notes:

Changelog

For all releases, please visit the Releases.

CategoryItemVersion
< 3.1.113.1.11
ConfigisUseLargeAngleModel
Scope: New apply
Only when userSelectedStitchingMode = 2 && maskStyle = 2 take effectAlso applies when userSelectedStitchingMode = 3
< 3.1.103.1.10
ConfiguserSelectedStitchingMode
Scope: New option
Available: 1 | 2Available: 1 | 2 | 3
Enable ultimate stitching mode when taking picture
maxRecordingSecs
Scope: New
-The maximum seconds for recording
Note: userSelectedStitchingMode = 3 is required to take effect.
allowLargeAngle
Scope: New
-Support setting of large angle when detection (0~90°)
minSkuSizeRatio
Scope: New
-Support setting of min SKU size ratio when detection(0.0~1.0)
< 3.1.83.1.8
DependenciesScope: RemovedDependencies declaration MUST include implementation 'fr.avianey.com.viewpagerindicator:library:2.4.1@aar'Removed
CompatibilityScope: Alignment-Support 16 KB memory page sizes
< 3.1.73.1.7
CallbackscancelTakePhotoCallback
Scope: New
-public void cancelTakePhotoCallback(String imagesPath)
< 3.1.63.1.6
Configlanguage
Scope: New
-Support setting the display language of the camera UI
< 3.1.53.1.5
ConfigmaskStyle
Scope: New option
Available: 3Available: 3 | 4
Enable different styles overlay when taking picture, 4 means product price-tag style
Note: product price-tag style equals 4 needs userSelectedStitchingMode = 2 to take effect.
< 3.1.43.1.4
ConfigenableGEOLocator
Scope: New
-Enable embedding longitude and latitude coordinates into the EXIF data of photo or not
< 3.1.23.1.2
ClassCameraConfig
Scope: route change
import com.clobotics.retail.stitch.utils.CameraConfig;import com.clobotics.retail.stitch.CameraConfig;
3.0.03.1.0
CallbackstakePhotoCallback -> takeSinglePhotoCallback
Scope: Function name、Parameters
public void takePhotoCallback(String imagePath, int imageIndex, String imageId, String pair, int taskId) public void takeSinglePhotoCallback(String imagePath, int imageIndex, String imageId, String pair)
endTakePhotoCallback
Scope: Parameters
public void endTakePhotoCallback(JSONObject stitchingInfo, int taskId, String groupTaskId, String stitchingPath)public void endTakePhotoCallback(String stitchingPath, JSONObject stitchingInfo)
startTakePhotoCallback
Scope: Delete
public int startTakePhotoCallback(String groupTaskId, int planId, int sceneId)-
ConfigmaskStyle
Scope: New
-Enable different styles overlay when taking picture, 3 means pure price-tag style
Note: pure price-tag style equals 3 needs userSelectedStitchingMode = 2 to take effect.

Powered by Clobotics Retail Team.