I had a problem to solve, we have applications built for platforms in Android, like GoogleTV, FireTv, Android Tablet, Firetablet and Android mobiles. It has a single codebase with modules separated for mobiles, tablets and TV. Again there is a differentiation in the build outputs, creating aab for GoogleTV, apk for FireTV, Android Mobiles and tablets. The application is written in Kotlin, the android build system is set up in Kotlin.

An Android application requires AndroidManifest file to list down the permissions and intents it wants to utilize. In my case I had to use android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK

which is available from SDK 34 onwards, and is mandatory to use this permission if we are using SDK 34.

This application has two variants the Firetablet and other Android mobiles and tablets. Firetablets were not all available for SDK 34, so I had to support lower versions also.

I had two approaches available to do this

a. Create another variant to differentiate between Firetablets SDK34 and higher and the rest of the platforms.

b. Try some hack within the Android manifest file and the gradle build system.

I chose the second one, and here I am outlining how this was done. Here are the steps

a. In Android manifest create a variable in the uses-permission section – <uses-permission android:name=”${fg_svc_media_perms}” />

b. Set the value of variable fg_svc_media_perms in the product flavor section. manifestPlaceholders[“fg_svc_media_perms”] = “android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK”

c. For another product flavor this can be set to blank.

I tested this and it works.

Here are the links that were useful to reach here.

https://stackoverflow.com/questions/23679999/conditionals-in-android-manifest

https://stackoverflow.com/questions/45170025/cant-use-manifest-placeholders-to-remove-a-permission

I always faced some issue or the other when I tried to set up a project in Android Studio, be it Kotlin or react-native. I have already mentioned some of the issues and how I solved them for react-native applications. In this post I will mention what all issues one can expect while configuring Android Studio for Kotlin projects. The information here holds good for both Windows and MacOS platforms. I have tested these solutions on Windows 10, Windows 11 and MacOS 13 Ventura.

Either we pickup a project from git sources, or open up a folder with source files, we need the project setup display the different build configurations. While we do this, we see a lot of initialization activity going on Android namely setting up the jdk, setting up gradle, project configuration etc. The final outcome is the Android IDE window where we can see in the top build section our build configuration selected. In the left side we have our product variants filled up based on whatever modules the project was built with.

This is an area errors occur for developers new to this platform. It is important to understand the dependencies, the build outputs, the caches and the sequence of operations. The best way to understand this is by listing down the dependencies required to build an Android App:

• JVM-JDK – This is the starting point, and most of our desktops have JAVA versions sprawling everywhere. In many situations Android studio will pick up a version based on what the system provides by default.
• Maven, gradle – These build tools have to be configured based on the JDK version we have. Improper configuration will lead to storage build errors. Gradle tool can be configured in two ways, we can have our own local gradle tool, or we can configure in our project using a gradle wrapper file which is located in gradle folder in the top level project. It has a gradle-wrapper.jar and gradle-wrapper.properties. Please note the distributionUrl. This is where gradle libraries are downloaded. This library has to be in sync with Java runtime we have.(https://docs.gradle.org/current/userguide/gradle_wrapper.html). In Android studio there is a reference to gradle user home. It is important to ensure all the common gradle files for use across projects are stored here and there is no conflict.

distributionBase=GRADLE_USER_HOME
distributionUrl=https\://services.gradle.org/distributions/gradle-7.2-bin.zip
distributionPath=wrapper/dists
zipStorePath=wrapper/dists
zipStoreBase=GRADLE_USER_HOME

• Android SDK -Android SDK has all the libraries that are responsible for processing the API calls make to the Andorra platform. This also has platform tools and emulator code binaries. It is required to ensure that this SDK, SDK tools. In short, the emulator, the platform-tools, tools and tools/bin has to be in system path for the platform commands to run. In the project folder, the file local.properties has the link to SDK location (the property sdk.dir maps to sdk location)
• Android Studio IDE structure, caches – Gradle installation sets up a gradle home folder where it downloads the plug in jar files required for the different projects that use gradle. Whenever calls to such library happen, the classes are first loaded from this cache. In order. to invalidate the cache, we can either delete this contents in this folder or we can use the “Invalidate Cache” option in Android Studio. Whenever we invalidate caches, all our projects that are loaded by Android Studio will download all the gradle distribution.
• kotlin plugins – Kotlin plugin versions are defined in the build.gradle file mentioned at project level. id 'org.jetbrains.kotlin.android' version '1.6.21' apply false.
• compose and other android frameworks – These will be part of project specific gradle dependency, these will not cause any issue with IDE bringing up the different flavours

Different Possibilities

Sometimes our gradle plugin files might reference Java 11 where as we might reference Java 1.8 in our project (I mean different versions). These have to be in sync. We need to check what our gradle JDK is being references in Settings/Build environment/gradle as well as the application references in build.gradle. It is always better to delete the .idea and .gradle files which are created by Studio, and do a fresh project sync with gradle. Sometimes it will be worthwhile to look at gradle settings provided in the Build output window, click on “Gradle Settings” link. You will get this by clicking on error message on the left side of split window pane.

An exception occurred applying plugin request [id: 'com.android.application']
> Failed to apply plugin 'com.android.internal.application'.
   > Android Gradle plugin requires Java 11 to run. You are currently using Java 1.8.
     You can try some of the following options:
       - changing the IDE settings.
       - changing the JAVA_HOME environment variable.
       - changing `org.gradle.java.home` in `gradle.properties`

--------------- My build.gradle file----------
compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }
    kotlinOptions {
        jvmTarget = '1.8'
    }

“Some Kotlin libraries attached to this project were compiled with a newer Kotlin compiler and can’t be read. please update Kotlin plugin”

Ensure that the compose version is in sync with Kotlin plugin version. Note the org.jetbrains.kotlin.android version.

buildscript {
    ext {
        compose_version = '1.0.1'
    }
}// Top-level build file where you can add configuration options common to all sub-projects/modules.
plugins {
    id 'com.android.application' version '7.1.1' apply false
    id 'com.android.library' version '7.1.1' apply false
    id 'org.jetbrains.kotlin.android' version '1.5.21' apply false
}

task clean(type: Delete) {
    delete rootProject.buildDir
}

Here I have tried to provide as many cases that can result in product flavors not appear in Android Studio IDE.

Gradle sync failed: This version of the Android Support plugin for IntelliJ IDEA (or Android Studio) cannot open this project

Try to change the distributionUrl in gradle-wrapper.properties to a lower version. If you still get an error with gradle versions not being in sync, downgrade the version plugins in build.gradle. e.g.

plugins {
id ‘com.android.application’ version ‘7.1.1’ apply false
id ‘com.android.library’ version ‘7.1.1’ apply false
id ‘org.jetbrains.kotlin.android’ version ‘1.6.21’ apply false
}

I will keep updating this as I go ahead and meet new scenarios. In case you find something not mentioned here, please let me know, I will add them here too.

A prerequisite to install any package involves an update of existing package manager, update its repository or package lists. It is not uncommon to get errors like the ones below, e.g the one below warns that we do not have a release file for ‘focal release’ for pulse audio. If we do not require this package, it will be good to remove this entry from the packages to be updated. Most of the issues can be possibly around ppa’s (personal package archives), as the name suggest these are users who can provide their own source code of libraries that can be included along with other libraries. This is done using launchpad which is the hosting platform for the free services.

sudo apt update && sudo apt upgrade
Reading package lists... Done
E: The repository 'http://ppa.launchpad.net/therealkenc/wsl-pulseaudio/ubuntu focal Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

In my case I wanted to stream media also over wsl, so I required pulse-sudio, but then this driver was not updated, so for the present I just removed it using the commands below. Whenever you hit a roadblock with the update not working, try to see if the download site is working and we are able to get the details. In this case I had to remove a ppa, which can be done by (https://askubuntu.com/questions/307/how-can-ppas-be-removed). As mentioned in the link the files are contained with .list extension the directory /etc/apt/sources.list.d. The idea of managing these lists is also mentioned here. https://itsfoss.com/repository-does-not-have-release-file-error-ubuntu/

sudo add-apt-repository --remove ppa:whatever/ppa

The next step is to install nvm using the install.sh script available for download from GitHub as raw downloadable files.

sudo add-apt-repository ppa:therealkenc/wsl-pulseaudio
sudo add-apt-repository --remove ppa:therealkenc/wsl-pulseaudio
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (35) error:140943E8:SSL routines:ssl3_read_bytes:reason(1000)

nvm install was failing with the above error. My ssl was set up properly, so not sure why this download errored. The workaround was to directly download the file or copy paste the contents of the file install.sh in the location https://github.com/nvm-sh/nvm/blob/master/install.sh locally and run them. nvm was installed. Once nvm install is done node install is straightforward. we can use an existing version, or install a new version.

Here is a good explanation from nvm team on how to install node or manage node with nvm. The npm is usually bundled and have the corresponding version.

$ nvm use 16
Now using node v16.9.1 (npm v7.21.1)
$ node -v
v16.9.1
$ nvm use 14
Now using node v14.18.0 (npm v6.14.15)
$ node -v
v14.18.0
$ nvm install 12
Now using node v12.22.6 (npm v6.14.5)
$ node -v
v12.22.6

Android push to playstore targetsdk upgrade

Google has mandated targetsdk 31 and any api below 31 cannot be deployed in play store. In this blog I have mentioned how I set up my development environment in MacOS, for a quick upgrade in 30 mins time. I developed an application using react native. Initially my application was set up with targetSDK 30. When I submitted this to play store, here is what I received as error message:

“Your app currently targets API level 30 and must target at least API level 31 to ensure it is built on the latest APIs optimized for security and performance. Change your app’s target API level to at least 31

Here’s the guide from google: https://developer.android.com/google/play/requirements/target-sdk#pre12“

A typical react native application has build scripts using grade as a build tool. Android Studio is the best way to setup the environment, though command line tools along with SDKManager can also be used to setup the environment. There are changes to be made to our development environment, our build files.

Here is the modification to gradle files. The build tools version and ndk have been chosen based on the latest that works with this combination. The minSDK version, can be changed to the requirement. the compileSdk version can be setup for higher targets as a matter of good practice. (build.gradle)

buildscript {
    ext {
        buildToolsVersion = "30.0.3"
        minSdkVersion = 21
        compileSdkVersion = 31
        targetSdkVersion = 31
        ndkVersion = "25.1.8937393"
    }

There are certain changes required to Android manifest also as we have to specify Intent behaviour as well.

<activity
        android:name=".MainActivity"
        android:label="@string/app_name"
        android:exported="true"
        android:configChanges="keyboard|keyboardHidden|orientation|screenSize|uiMode"
        android:launchMode="singleTask"
        android:windowSoftInputMode="adjustResize">

The next set of changes are required to set up our environment in place. This SDK requires a Java upgrade as compared to earlier versions, we required Java 8, but here we require Java 11. So following changes are required

JAVA_HOME environment variable to point to java 11

The system path should pick up Java 11 SDK binaries. For MacOS I used home-brew to install openjdk11.

Taken form the link - https://formulae.brew.sh/formula/openjdk@11
https://github.com/Homebrew/discussions/discussions/2405

For the system Java wrappers to find this JDK, symlink it with
  sudo ln -sfn /usr/local/opt/openjdk@11/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk-11.jdk

openjdk@11 is keg-only, which means it was not symlinked into /usr/local,
because this is an alternate version of another formula.

If you need to have openjdk@11 first in your PATH, run:
  echo 'export PATH="/usr/local/opt/openjdk@11/bin:$PATH"' >> ~/.zshrc

For compilers to find openjdk@11 you may need to set:
  export CPPFLAGS="-I/usr/local/opt/openjdk@11/include"

The error below was resolved once Java version was changes

> Task :react-native-device-info:compileDebugJavaWithJavac FAILED

143 actionable tasks: 20 executed, 123 up-to-date

An exception has occurred in the compiler (1.8.0_311). Please file a bug against the Java compiler via the Java bug reporting page (http://bugreport.java.com) after checking the Bug Database (http://bugs.java.com) for duplicates. Include your program and the following diagnostic in your report. Thank you.

java.lang.AssertionError: annotationType(): unrecognized Attribute name MODULE (class com.sun.tools.javac.util.UnsharedNameTable$NameImpl)

If android manifest does not have declarations for intent, the error below might be seen.

error Failed to install the app. Make sure you have the Android development environment set up: https://reactnative.dev/docs/environment-setup.

Error: Command failed: ./gradlew app:installDebug -PreactNativeDevServerPort=8081

/Users/../AndroidManifest.xml Error:

Apps targeting Android 12 and higher are required to specify an explicit value for `android:exported` when the corresponding component has an intent filter defined. See https://developer.android.com/guide/topics/manifest/activity-element#exported for details.

All the above errors are seen when we do an:

npm start 
npm run android

For one last time I got an out of memory error, but then this was a one off situation and got resolved automatically in the second run.

Execution failed for task ‘:app:packageDebug’.

> A failure occurred while executing com.android.build.gradle.tasks.PackageAndroidArtifact$IncrementalSplitterRunnable

   > java.lang.OutOfMemoryError (no error message)

After this I was able to get my app to run. I hope this works for you too. Let me know if you have issues.

——————–

Links referred

https://stackoverflow.com/questions/68387270/android-studio-error-installed-build-tools-revision-31-0-0-is-corrupted

https://stackoverflow.com/questions/67412084/android-studio-error-manifest-merger-failed-apps-targeting-android-12

https://stackoverflow.com/questions/68344424/unrecognized-attribute-name-module-class-com-sun-tools-javac-util-sharednametab

https://developer.android.com/studio/command-line/sdkmanager

https://docs.oracle.com/en/java/javase/11/install/installation-jdk-macos.html#GUID-E8A251B6-D9A9-4276-ABC8-CC0DAD62EA33

https://formulae.brew.sh/formula/openjdk@11

While setting up rails project we mostly use home-brew to install some of our dependent components like Postgres, or redis. Sometimes brew update or upgrade results in breaking o some of our existing applications, services and components. Here I am listing just a table of some of the errors, with a timeline, we can use to see how relevant the problem or solution is to the one we have now.

As part of setting up Opentelemetry in my rust application and test it locally, I chose to use Docker-Desktop, its Kubernetes environment and my rust application, all running locally. The part of setting up my application to send traces directly to Jaeger, deployed as a docker was easy. This just required the docker port to be set up as 4217, the default collector port, and expose it to the application using the docker commands. This becomes tricky when I had to set up a collector in between that can forward requests to Jaeger. I needed all of them in a single dns boundary. I chose Kubernetes over Docker Swarm to do this with no specific reason to start with. Now I was looking for Jaeger configuration files for Kubernetes, Otel-Collector configuration for Kubernetes. There was a difficulty in configuring and deploying both of them. Here are few of the changes I made to make them work. Both Jaeger and Opentelemetry collector are deployed in the observability namespace. I used helm charts to deploy Jaeger, and kubectl to directly deploy open telemetry collector. The steps to deploy Jaeger has two steps, deploy the Jaeger operator and then deploy Jaeger.

helm repo add jetstack https://charts.jetstack.io
helm repo add jaegertracing https://jaegertracing.github.io/helm-charts

helm install \
  cert-manager jetstack/cert-manager \
  --namespace observability \
  --create-namespace \
  --version v1.8.0 \

helm install myrelease jaegertracing/jaeger-operator -n observability

helm install jaeger-all-in-one jaeger-all-in-one/jaeger-all-in-one -n observability

Jaeger also requires cert-manager to be installed, which is the line 3, and then we will have to port forward set up before Jaeger UI is deployed. for additional details please refer https://cert-manager.io/docs/installation/helm/#option-2-install-crds-as-part-of-the-helm-release

There were issues using jaegertracing/jaeger charts as it was giving errors like “…ensure CRDs are installed first…”. So I chose jaeger-all-in-one.

Next we need to install opentelemetry-collector. I am deploying it in the same namespace “observability”. we will need to modify the standard open telemetry-collector yaml, to point to the pods name of jaeger-all-in-one e.g.

Here is where I downloaded the open-telemetry collector Yaml from – https://github.com/open-telemetry/opentelemetry-collector/blob/main/examples/k8s/otel-config.yaml. The 14250 port is being used to send messages using jaeger.

Change 1

    exporters:
      jaeger:
        endpoint: "jaeger-all-in-one:14250" # Replace with a real endpoint.
        tls:
          insecure: true

Change 2

service:
      extensions: [zpages, memory_ballast]
      pipelines:
        traces:
          receivers: [otlp]
          processors: [memory_limiter, batch]
          exporters: [jaeger]

Now this can be loaded using kubectl in the observability namespace.

kubectl apply -f Otel-collector.yaml -n observability

After this we need to set up few port forwarding, which can be done as follows, this is strictly for local development sandboxes and not a secure production implementation

kubectl --namespace observability port-forward $POD_NAME 16686:16686
kubectl --namespace observability port-forward $POD_NAME 4317:4317

These can be spawned in 2 different terminals. After that we can load the UI in a browser http://127.0.0.1:16686 and let us start our app to send the traces to the open telemetry collector. Though this Otel-collector.yaml deploys Otel-agent as a daemonset, I directly sent requests to collector which is deployed as a service on port 4317. The collector is internally configured to forward requests to Jaeger collector on 14250 as grpc.

This blog will be useful for developers new to the concept of scheduling with Sidekiq. Sidekiq framework helps us to run ruby jobs in background, more about this is available here. A typical Sidekiq job set up will require a job definition (greet_all.rb), which is called within an implementation of a Sidekiq worker (greet_all_worker.rb). This worker is scheduled in a file called schedule.yml. This location of this file is loaded in Sidekiq as an initialiser (Sidekiq.rb). Here is how some of these files will look like.

#apps/models/greet_all.rb
class GreetAll
   def self.greet
     print "Hello All"
   end
end

#apps/workers/greet_all_worker.rb
class GreetAllWorker
  include Sidekiq::Worker

  sidekiq_options retry: true

  def perform(*args)
    GreetAll.greet
  end
end

#create config/schedule.yml
process_greet_all_worker:
  cron: "0 0 23 * * * Asia/Kolkata" #if required to run at 11pm IST
  #cron: "*/2 * * * *" #if required to run every 2 mins
  class: "GreetAllWorker"
  queue: low

#Add to initializers/sidekiq.rb
schedule_file = 'config/schedule.yml'

if File.exist?(schedule_file) && Sidekiq.server?
  Sidekiq::Cron::Job.load_from_hash YAML.load_file(schedule_file)
end

Debugging this program can be done at 3 levels:

The first level is one where we directly check how our program is working. This can be done using rails console, and calling the method in our case GreetAll.greet.

Next we can try to see if our files and schedule is set up properly (an entire call triggered by system works fine). This can be done by scheduling a run at a defined interval. The commented line of the cron allows the process to be called every 2 mins.

Once we are through with this, the final test is to verify if the cron scheduling is proper. This is also easy but the only issue is that the documentation around cron is a little misleading. In most of the places, I have seen the * * * * * * (6 stars) to denote minute, hour, day, week and so on. but then this did not work for me. It starts with seconds, minutes, hour, day and so on.

The best way to understand this is to use the library that ruby uses to parse the cron and date times. Fugit is the library used by Sidekiq, and it provides an easy way to check what our configuration means. We can use rails console to try the same. Let us feed our cron information to the Fugit library as below. We can verify this information and then we are ready to ship our code.

>require 'fugit'
=> false
>c = Fugit::Cron.parse('0 29 11 * * * America/Los_Angeles')
=> #<Fugit::Cron:0x00007fd6fa335698 @original="0 29 11 * * * America/Los_Angeles", @cron_s=nil, @seconds=[0], @minutes=[29], @hours=[11], @monthdays=nil, @months=nil, @weekdays=nil, @zone="America/Los_Angeles",

One point to note is that we will need to clear our redis cache, the moment a change is made to schedule.yml files. The cron jobs will then be reloaded into the database. An easy way to do this is

redis-cli flushall

Some useful links are

https://medium.com/serpapi/ruby-schedulers-whenever-vs-sidekiq-cron-vs-sidekiq-scheduler-b229d7ca5256

https://github.com/floraison/fugit

https://stackoverflow.com/questions/24886371/how-to-clear-all-the-jobs-from-sidekiq

https://github.com/floraison/fugit