Compiling for Android

Moderators: Elijah, Igbo, timetraveller

Archimedes

Android Engines Top Active Users
Forum Contributions
Points: 42 582,00 
Posts: 2059
Joined: 04/11/2019, 21:13
Status: Offline (Active 7 Hours, 59 Minutes ago)
Medals: 2
Topics: 158
Reputation: 7111
Been thanked: 6477 times

Compiling for Android

Post by Archimedes »

When using Android NDK, there are three ways to produce android builds: ndk-build, cmake or standalone toolchains.

The following (very short) description will show you, how it works with (the command line tool) ndk-build. I'm using ndk-build, because i'm not delivering a software (app) for different environments, i'm only interested in compiling a source code (c or c++) to android. No more, no less.

But before we can start, we need the Android NDK. It can be downloaded from https://developer.android.com/ndk/downloads. Choose the right version for you. Unzip the zip archive to a directory of your choice and put the root directory of your android-ndk directory to your search path (the command line tool ndk-build should start from the command line in every directory). In Windows you enter the path to the system or user path, in Linux you may add the following lines at the end of the file „.bashrc“ in your home directory (assuming you unzip the content of the zip archiv to usr/lib/android-ndk), so the path is active, when you start the terminal.

Code: Select all

export ANDROID_NDK_HOME=/usr/lib/android-ndk
export PATH=$PATH:$ANDROID_NDK_HOME
OK, now we are ready to go. The following examples works in Windows and in Linux.

Let us begin with an current project, which is easy to compile: Demolito. :smile:

1. Download the sources from https://github.com/lucasart/Demolito. If you donwload the sources as zip archive, unzip the files to a directory of your choice (e. g. Demolito).

2. When you go in to the root directory of Demolito, you will see a src directory. Create a directory named jni in the root directory of Demolito and put the following file in to that directory.

Android.mk:

Code: Select all

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := Demolito
LOCAL_SRC_FILES := $(wildcard ../src/*.c)
LOCAL_CFLAGS += -DVERSION=\"2020-10-10\"
include $(BUILD_EXECUTABLE)
As you can see, you need only one file, named Android.mk, and a few lines of code to make Android builds with ndk-build from Android NDK. All lines above are necessary. For further information, what they means, please inform you at https://developer.android.com/ndk/.

For us, line 3, 4 and 5 are necessary.

Line 3: Here you define a name for your builds (executables). In our case we choose the name Demolito.
Line 4: Here are all source files listed, which have to be compiled. In our case we want to compile all *.c files in the src directory.
Line 5: Here you can define additional compiler flags for the compiling process. In your case we only need the parameter „VERSION“, which is used in the source code of Demolito for showing the correct version number. No more parameter needed. Android NDK, i mean ndk-build, will do the rest (in most cases, ndk-build choose the right parameter for you.)

3. Start the command line and change to the jni directory and call ndk-built. The compiling process is starting and when there is no error, the executables will be found in the libs directory.

4. Ready. popcorm1

OK, this is a very basic example, which works with the default settings of Android NDK.

Normally, we have to add a few more compiler settings. Let us describe some of them with the Demolito example.

When we take a look at the makefile in the src directory, we can see, that the compiler language of the project is gnu11. Normally ndk-build find this out automatically for you, but in some cases, this won‘t work. So we add this information to the Android.mk file, so the compiler knows, in what compiler language it should compile the project.

Android.mk:

Code: Select all

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := Demolito
LOCAL_SRC_FILES := $(wildcard ../src/*.c)
LOCAL_CFLAGS += -std=gnu11 -DVERSION=\"2020-10-10\"
include $(BUILD_EXECUTABLE)
Also, when we want some optimizations regarding runtime performance, we can add additional code optimizations with the -O3 compiler flag (default with ndk-build is -O2). This will give us a very small speed up. If you want a bigger speed up, you can use link time optimization with the compiler and linking flag -flto. To make the story short, here the resulted file.

Android.mk:

Code: Select all

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := Demolito
LOCAL_SRC_FILES := $(wildcard ../src/*.c)
LOCAL_CFLAGS += -std=gnu11 -O3 -flto -DVERSION=\"2020-10-10\"
LOCAL_LDFLAGS += -flto
include $(BUILD_EXECUTABLE)
With this configuration you will receive compatible and fast compiles of Demolito for Android devices. Further optimizations can be don with profile guide optimizations (pgo) and so on. But be carefull with additional (optimized) compiler flags. wink

For completeness, i‘m using the following Android.mk file for compiling Demolito.

Code: Select all

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := Demolito
LOCAL_SRC_FILES := $(wildcard ../src/*.c)
LOCAL_CFLAGS += -std=gnu11 -DNDEBUG -O3 -flto -Wfatal-errors -Wall -Wextra -Wshadow -DVERSION=\"2020-10-10\"
LOCAL_LDFLAGS += -flto
include $(BUILD_EXECUTABLE)
The -W flags will show you only information during the build process. Normally they only necessary for the developer, so they can see, what code they could optimize and so on.

With ndk-build you have not only the file Android.mk for setting compiling options, there is also a file named Application.mk which can be used to tell the compiler additional instructions.

For example, if you want, that your builds are compatible with the Android API level 16, you have to put the following line in to Application.mk (which should be also in the jni directory).

Application.mk:

Code: Select all

APP_PLATFORM := android-16
However, this setting is normally not necessary, because it‘s the default setting, when using ndk-build. wink

This example shows, all what you need for compiling to Android is, an installed Android NDK (root directory should be in the search path), two files, named Android.mk and Application.mk, with the correct settings, in the jni directory. The only thing you have to do is, calling ndk-build from the command line within the jni directory. yes3

Keep in mind, this is only a very short and by far not complete description, but it shows the idea behind of cross compiling with ndk-build from Android NDK.
Archimedes

Android Engines Top Active Users
Forum Contributions
Points: 42 582,00 
Posts: 2059
Joined: 04/11/2019, 21:13
Status: Offline (Active 7 Hours, 59 Minutes ago)
Medals: 2
Topics: 158
Reputation: 7111
Been thanked: 6477 times

Compiling for Android

Post by Archimedes »

Here are some examples of Android.mk and Application.mk from another projects.

Phalanx needs the following settings.

Android.mk:

Code: Select all

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := Phalanx
LOCAL_SRC_FILES := $(wildcard ../src/*.c)
LOCAL_CFLAGS += --std=gnu89 -DNDEBUG -O3 -flto -Wfatal-errors -Wall -Wextra -Wshadow
LOCAL_LDFLAGS += -flto
include $(BUILD_EXECUTABLE)
Very similar to Demolito. Only the name of the executables and the compiler language has changed. A special version parameter, as used in the source code of Demolito, is not necessary.

Application.mk:

Code: Select all

APP_PLATFORM := android-16
Nothing special here.

Now let‘s see, how the files looks like for the Texel project, which is written in the c++ language.

Android.mk:

Code: Select all

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := Texel
LOCAL_C_INCLUDES := ../lib/texellib ../lib/texellib/gtb/compression ../lib/texellib/gtb/compression/lzma ../lib/texellib/gtb/sysport
LOCAL_SRC_FILES := $(wildcard ../app/texel/*.cpp ../lib/texellib/*.cpp ../lib/texellib/util/*.cpp ../lib/texellib/syzygy/*.cpp ../lib/texellib/gtb/*.c ../lib/texellib/gtb/sysport/*.c ../lib/texellib/gtb/compression/*.c ../lib/texellib/gtb/compression/lzma/*.c)
LOCAL_CFLAGS += -DNDEBUG -O3 -flto -fexceptions -Wfatal-errors -Wall -Wextra -Wshadow
LOCAL_LDFLAGS += -flto
include $(BUILD_EXECUTABLE)
As you can see in line 5 (LOCAL_SRC_FILES), you have a lot of more locations, where the source files are located (the compiler must know them). And in line 4 (LOCAL_C_INCLUDES) you listed all the include directories. The compiler sometimes not find additional files (mostly header files) during compilation process. This is the right place to tell the compiler, where to find them. In line 6 (LOCAL_CFLAGS) you will find an new parameter -fexceptions. This is because the source code contains exceptions and the parameter tells the compiler to use them, otherwise it will bring up an error.

Application.mk:

Code: Select all

APP_PLATFORM := android-16
APP_STL := c++_static
Last line is needed for c++, it tells the compiler to include the required libc++ library as static library.
Archimedes

Android Engines Top Active Users
Forum Contributions
Points: 42 582,00 
Posts: 2059
Joined: 04/11/2019, 21:13
Status: Offline (Active 7 Hours, 59 Minutes ago)
Medals: 2
Topics: 158
Reputation: 7111
Been thanked: 6477 times

Compiling for Android

Post by Archimedes »

In the meanwhile, i'm using always the same structure in the files Android.mk and Application.mk. The layout of the two files are always the same. So it's easier for me, when dealing with different chess engines.

As an example, see, how the files looks like for Demolito now.

Android.mk:

Code: Select all

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := Demolito
LOCAL_C_INCLUDES := 
LOCAL_SRC_FILES := $(wildcard ../src/*.c)
LOCAL_CFLAGS += -std=gnu11 -DNDEBUG -O3 -flto -fno-stack-protector -fomit-frame-pointer -Wfatal-errors -Wall -Wextra -Wshadow -DVERSION=\"2020-10-10\"

ifeq ($(TARGET_ARCH_ABI), arm64-v8a)
  LOCAL_CFLAGS += 
else
  ifeq ($(TARGET_ARCH_ABI), armeabi-v7a)
    LOCAL_CFLAGS += 
  else
    ifeq ($(TARGET_ARCH_ABI), x86)
      LOCAL_CFLAGS += -mssse3 -mfpmath=sse -m32
    else
      ifeq ($(TARGET_ARCH_ABI), x86_64)
        LOCAL_CFLAGS += -msse4.2 -mpopcnt -m64
      endif
    endif
  endif
endif

LOCAL_LDFLAGS += -flto
include $(BUILD_EXECUTABLE)
As you can see, i'm using -O3, -flto, -fno-stack-protector and -fomit-frame-pointer as target independent compiler settings now and i have an extra if-then construction for dealing with different architecture compiler settings. The if-then construct you see, is the same for all chess engines. Some chess engines needs extra compiler definitions (e. g. -DUSE_NEON), some not (like Demolito).

Application.mk:

Code: Select all

APP_ABI := arm64-v8a armeabi-v7a x86 x86_64
APP_PLATFORM := android-21
APP_STL := 
Here is an example, how the files looks like with the Scorpio engine.

Android.mk:

Code: Select all

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := Scorpio
LOCAL_C_INCLUDES := 
LOCAL_SRC_FILES := $(wildcard ../src/*.cpp)
LOCAL_CFLAGS += -std=c++11 -DNDEBUG -O3 -flto -fno-stack-protector -fomit-frame-pointer -Wfatal-errors -Wall -Wextra -Wshadow -DPARALLEL -DUSE_SPINLOCK

ifeq ($(TARGET_ARCH_ABI), arm64-v8a)
  LOCAL_CFLAGS += -DARC_64BIT -DHAS_POPCNT
else
  ifeq ($(TARGET_ARCH_ABI), armeabi-v7a)
    LOCAL_CFLAGS += 
  else
    ifeq ($(TARGET_ARCH_ABI), x86)
      LOCAL_CFLAGS += -mssse3 -mfpmath=sse -m32 -DHAS_PREFETCH
    else
      ifeq ($(TARGET_ARCH_ABI), x86_64)
        LOCAL_CFLAGS += -msse4.2 -mpopcnt -m64 -DARC_64BIT -DHAS_POPCNT -DHAS_PREFETCH
      endif
    endif
  endif
endif

LOCAL_LDFLAGS += -flto
include $(BUILD_EXECUTABLE)
The compiler definitions -DPARALLEL and -DUSE_SPINLOCK are used for all architectures (arm64-v8a, armeabi-v7a, x86 and x86_64). Architecture dependant compiler definitions are in the if-then construction. For example, -DHAS_PREFETCH can only be used for the x86 and x86_64 architecture.

Application.mk:

Code: Select all

APP_ABI := arm64-v8a armeabi-v7a x86 x86_64
APP_PLATFORM := android-21
APP_STL := c++_static
Now, you have an idea, how you can compile with Android NDK and ndk-build for Android devices. It's just a way of many others. But when you get the trick, you can do it with other solutions too, as you are able to deal with compiler settings now.
Archimedes

Android Engines Top Active Users
Forum Contributions
Points: 42 582,00 
Posts: 2059
Joined: 04/11/2019, 21:13
Status: Offline (Active 7 Hours, 59 Minutes ago)
Medals: 2
Topics: 158
Reputation: 7111
Been thanked: 6477 times

Compiling for Android

Post by Archimedes »

On Linux, you don't need the environment variable ANDROID_NDK_HOME as described above. You only need to add the following line at the end of .bashrc (if android-ndk is located on a different place, change the path).

Code: Select all

export PATH=$PATH:/usr/lib/android-ndk
SwiftSnips
Forum Contributions
Points: 6 000,00 
Posts: 139
Joined: 03/12/2020, 21:06
Status: Offline (Active 2 Months, 1 Week, 2 Days, 23 Hours, 21 Minutes ago)
Topics: 0
Reputation: 18
Been thanked: 33 times

Compiling for Android

Post by SwiftSnips »

Here are some examples from Xiphos

Code: Select all

 APP_ABI := arm64-v8a armeabi-v7a x86 x86_64
APP_PLATFORM := android-21
APP_STL := 
 

Code: Select all

 LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := Xiphos
LOCAL_C_INCLUDES := 
LOCAL_SRC_FILES := $(wildcard ../src/*.c) ../src/fathom/tbprobe.c
LOCAL_CFLAGS += -DNDEBUG -O3 -flto -fno-stack-protector -fomit-frame-pointer -Wfatal-errors -Wall -Wextra -Wshadow -D_NOPOPCNT

ifeq ($(TARGET_ARCH_ABI), arm64-v8a)
  LOCAL_CFLAGS += 
else
  ifeq ($(TARGET_ARCH_ABI), armeabi-v7a)
    LOCAL_CFLAGS += 
  else
    ifeq ($(TARGET_ARCH_ABI), x86)
      LOCAL_CFLAGS += -mssse3 -mfpmath=sse -m32
    else
      ifeq ($(TARGET_ARCH_ABI), x86_64)
        LOCAL_CFLAGS += -msse4.2 -mpopcnt -m64
      endif
    endif
  endif
endif

LOCAL_LDFLAGS += -flto
include $(BUILD_EXECUTABLE)
Walkot
Forum Contributions
Points: 9 587,00 
Posts: 41
Joined: 25/01/2020, 7:24
Status: Offline (Active 9 Months, 4 Weeks, 1 Day, 22 Hours, 17 Minutes ago)
Topics: 1
Reputation: 14
Been thanked: 17 times

Compiling for Android

Post by Walkot »

Is it possible to make console applications: for calculating the rating (bayeselo or ordo), for solving chess problems (popeye)?
Skynet
Points: 0,00 
Status: Offline (Active 3 Hours, 16 Minutes ago)
Topics: 

Compiling for Android

Post by Skynet »

Walkot wrote:Is it possible to make console applications: for calculating the rating (bayeselo or ordo), for solving chess problems (popeye)?
Ordo 1.2.6 ARM8 for android already exist (compiled by hagtorp). I haven't used it before i'm using Ordo on my PC, but description from command file says -> Name of pgn file must be Results.pgn, pgn file must be in the current directory ->
/storage/emulated/0/Results.pgn (directory and pgn's name can be changed in ordo.txt document which is in ordo-126 folder).
arm8 ordo-126 https://pixeldrain.com/u/32R94PhG
Archimedes

Android Engines Top Active Users
Forum Contributions
Points: 42 582,00 
Posts: 2059
Joined: 04/11/2019, 21:13
Status: Offline (Active 7 Hours, 59 Minutes ago)
Medals: 2
Topics: 158
Reputation: 7111
Been thanked: 6477 times

Compiling for Android

Post by Archimedes »

Walkot wrote:Is it possible to make console applications: for calculating the rating (bayeselo or ordo), for solving chess problems (popeye)?
https://www.remi-coulom.fr/Bayesian-Elo

Not tested. If it works and if there is a newer version available, please let me know.

Bayeselo 0057:
https://app.box.com/s/ae5bl9iv6jhqkpuw7zep6xakkl9vemrw
Archimedes

Android Engines Top Active Users
Forum Contributions
Points: 42 582,00 
Posts: 2059
Joined: 04/11/2019, 21:13
Status: Offline (Active 7 Hours, 59 Minutes ago)
Medals: 2
Topics: 158
Reputation: 7111
Been thanked: 6477 times

Compiling for Android

Post by Archimedes »

For c++ sources i always uses the files Android.mk and Application.mk from the Fruit engine as a starting point, as templates. For c sources i always uses the files Android.mk and Application.mk from OliThink engine as templates.

Bayeselo is written in c++. Therefore, i used the files Android.mk and Application.mk from the fruit engine as templates and made the necessary changes.

Bayeselo needs the following changes in version.cpp for proper compiling with Android NDK.

Before:

Code: Select all

"version "VERSION", Copyright (C) 1997-2010 Remi Coulom.\n"
After:

Code: Select all

"version " VERSION ", Copyright (C) 1997-2010 Remi Coulom.\n"
Before:

Code: Select all

"compiled "__DATE__" "__TIME__".\n"
After:

Code: Select all

// "compiled "__DATE__" "__TIME__".\n"
Before:

Code: Select all

"version "VERSION".\n"
After:

Code: Select all

"version " VERSION ".\n"
Archimedes

Android Engines Top Active Users
Forum Contributions
Points: 42 582,00 
Posts: 2059
Joined: 04/11/2019, 21:13
Status: Offline (Active 7 Hours, 59 Minutes ago)
Medals: 2
Topics: 158
Reputation: 7111
Been thanked: 6477 times

Compiling for Android

Post by Archimedes »

I've updated all zip archives containing engines with nnue support.

Chess Engines with nnue support:

Code: Select all

Crystal
Igel
Marvin
Mayhem
Minic
Nemorino
RubiChess
Scorpio
Seer
Stockfish
SugaR AI
SugaR AI ICCF
Vajolet2
In the new directory networks there are now a readme.txt file and the default net for the engine. If an engine has no own neural network file, i just copy the default net from Stockfish to that directory. Scorpio has only a readme.txt file in the networks directory.

Don't know, wether it is a good idea, die to the file sizes. May be, it is enough, when there is only a readme.txt file inside with further instructions.
Walkot
Forum Contributions
Points: 9 587,00 
Posts: 41
Joined: 25/01/2020, 7:24
Status: Offline (Active 9 Months, 4 Weeks, 1 Day, 22 Hours, 17 Minutes ago)
Topics: 1
Reputation: 14
Been thanked: 17 times

Compiling for Android

Post by Walkot »

Just now I saw Bayeselo. But there is also a newer version 0058:https://github.com/MichaelB7/bayeselo.Everything works mst , but it was not possible to start it correctly right away xrf03 . Here is a quick guide [xrf32 :1) download SManager 2)download the above archive Bayeselo 3)In SManager, copy the desired version from the archive and your pgn file 4) Click the house, then "DATA home" and paste the selected files. 5)at the top right click "three dots", console, Run shell 6)enter cd /data/user/0/os.tools.scriptmanager/ 7)enter ./Bayeselo 8)The program works! 9) Instructions for working with the program on the Internet, and the list of commands is called by the"? " sign. cld
Walkot
Forum Contributions
Points: 9 587,00 
Posts: 41
Joined: 25/01/2020, 7:24
Status: Offline (Active 9 Months, 4 Weeks, 1 Day, 22 Hours, 17 Minutes ago)
Topics: 1
Reputation: 14
Been thanked: 17 times

Compiling for Android

Post by Walkot »

How do I still run ndk-build in Windows 10 ? Connecting the android-ndk folder to PATH in the system settings did not bring any results. Download and run the Cygwin - also did not work. Reports that ndk-build is not present in any form. What to do??? 35a
Archimedes

Android Engines Top Active Users
Forum Contributions
Points: 42 582,00 
Posts: 2059
Joined: 04/11/2019, 21:13
Status: Offline (Active 7 Hours, 59 Minutes ago)
Medals: 2
Topics: 158
Reputation: 7111
Been thanked: 6477 times

Compiling for Android

Post by Archimedes »

You extracted the zip archive to a folder (with no spaces in the folder names)? Then you must set the system path variable to that path. For example, if you extracted Android NDK to C:\android-ndk, then you must add C:\android-ndk to your system path variable (https://www.java.com/en/download/help/path.html). Start the command line and enter ndk-build. When it comes an error message directly from Android NDK, then you are ready to go.
Walkot
Forum Contributions
Points: 9 587,00 
Posts: 41
Joined: 25/01/2020, 7:24
Status: Offline (Active 9 Months, 4 Weeks, 1 Day, 22 Hours, 17 Minutes ago)
Topics: 1
Reputation: 14
Been thanked: 17 times

Compiling for Android

Post by Walkot »

I realized my mistake! xrf16 It's not about the PATH , it's about the file Android.mk. I made it as .txt was also replaced with .mk in the name, but the system did not perceive it as .mk. In general, I had to take the original. mk (from any assembly), delete the old text in a regular notepad and write down what you need. The file still remains as .mk, although it is changed in the text editor.
Archimedes

Android Engines Top Active Users
Forum Contributions
Points: 42 582,00 
Posts: 2059
Joined: 04/11/2019, 21:13
Status: Offline (Active 7 Hours, 59 Minutes ago)
Medals: 2
Topics: 158
Reputation: 7111
Been thanked: 6477 times

Compiling for Android

Post by Archimedes »

I updated all engines, written in Go (Combusken, Counter, Donna, Dragontooth, FrankyGo, GopherCheck and Zurichess). Most of them should work now on arm64-v8a, armeabi-v7a, x86 and x86_64 devices.
Post Reply

Return to “Programming, Technical Discussions, Chess related questions etc.”