20201022通过PlatformIO刷入MicroPython遇到的问题

续上回,修改Makefile,将ESPIDF_LWIP_O也编入静态库中

staticlib:
	$(ECHO) "LIB micropython lib"
	# $(Q)$(AR) rsc $(BUILD)/libmicropython.a $(OBJ)
	$(Q)$(AR) rsc $(BUILD)/libmicropython.a $(OBJ) $(ESPIDF_LWIP_O:%.o=build-GENERIC%.o)

libmicropython.a链接到pio中时发现一些函数出现重复定义,pio已经将lwip编译成静态库了。

分别查看micropython repo中编译的liblwip.a和pio编译的liblwip.a的符号表

$ readelf -s liblwip.a | grep "pppapi" -C 10

micropython repo中的结果:

File: liblwip.a(pppapi.o)

Symbol table '.symtab' contains 59 entries:
   Num:    Value  Size Type    Bind   Vis      Ndx Name
     0: 00000000     0 NOTYPE  LOCAL  DEFAULT  UND
     1: 00000000     0 FILE    LOCAL  DEFAULT  ABS pppapi.c
     2: 00000000     0 SECTION LOCAL  DEFAULT   29
     3: 00000000     0 SECTION LOCAL  DEFAULT   30
     4: 00000000     0 SECTION LOCAL  DEFAULT   31
     5: 00000000     0 SECTION LOCAL  DEFAULT   32
     6: 00000000    18 FUNC    LOCAL  DEFAULT   32 pppapi_do_ppp_set_default
     7: 00000000     0 SECTION LOCAL  DEFAULT   34
     8: 00000000    22 FUNC    LOCAL  DEFAULT   34 pppapi_do_ppp_set_auth
     9: 00000000     0 SECTION LOCAL  DEFAULT   36
    10: 00000000    23 FUNC    LOCAL  DEFAULT   36 pppapi_do_pppos_create
    11: 00000000     0 SECTION LOCAL  DEFAULT   38
    12: 00000000    19 FUNC    LOCAL  DEFAULT   38 pppapi_do_ppp_connect
    13: 00000000     0 SECTION LOCAL  DEFAULT   40
    14: 00000000    19 FUNC    LOCAL  DEFAULT   40 pppapi_do_ppp_close
    15: 00000000     0 SECTION LOCAL  DEFAULT   42
    16: 00000000    17 FUNC    LOCAL  DEFAULT   42 pppapi_do_ppp_free
    17: 00000000     0 SECTION LOCAL  DEFAULT   44
    18: 00000000    21 FUNC    LOCAL  DEFAULT   44 pppapi_do_ppp_ioctl
    19: 00000000     0 SECTION LOCAL  DEFAULT   46
    20: 00000000     0 SECTION LOCAL  DEFAULT   48
    21: 00000000     0 SECTION LOCAL  DEFAULT   50
    22: 00000000     0 SECTION LOCAL  DEFAULT   52
    23: 00000000     0 SECTION LOCAL  DEFAULT   54
    24: 00000000     0 SECTION LOCAL  DEFAULT   56
    25: 00000000     0 SECTION LOCAL  DEFAULT   58
    26: 00000000     0 SECTION LOCAL  DEFAULT    1
    27: 00000000     0 SECTION LOCAL  DEFAULT    3
    28: 00000000     0 SECTION LOCAL  DEFAULT    5
--
    41: 00000000     0 SECTION LOCAL  DEFAULT   61
    42: 00000000     0 SECTION LOCAL  DEFAULT   62
    43: 00000000     0 SECTION LOCAL  DEFAULT   64
    44: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND netif_set_default
    45: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND ppp_set_auth
    46: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND pppos_create
    47: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND ppp_connect
    48: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND ppp_close
    49: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND ppp_free
    50: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND ppp_ioctl
    51: 00000000    21 FUNC    GLOBAL DEFAULT   46 pppapi_set_default
    52: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND tcpip_api_call
    53: 00000000    26 FUNC    GLOBAL DEFAULT   48 pppapi_set_auth
    54: 00000000    30 FUNC    GLOBAL DEFAULT   50 pppapi_pppos_create
    55: 00000000    25 FUNC    GLOBAL DEFAULT   52 pppapi_connect
    56: 00000000    25 FUNC    GLOBAL DEFAULT   54 pppapi_close
    57: 00000000    21 FUNC    GLOBAL DEFAULT   56 pppapi_free
    58: 00000000    26 FUNC    GLOBAL DEFAULT   58 pppapi_ioctl

pio中的结果:

File: liblwip.a(pppapi.c.o)

Symbol table '.symtab' contains 12 entries:
   Num:    Value  Size Type    Bind   Vis      Ndx Name
     0: 00000000     0 NOTYPE  LOCAL  DEFAULT  UND
     1: 00000000     0 FILE    LOCAL  DEFAULT  ABS pppapi.c
     2: 00000000     0 SECTION LOCAL  DEFAULT    1
     3: 00000000     0 SECTION LOCAL  DEFAULT    2
     4: 00000000     0 SECTION LOCAL  DEFAULT    3
     5: 00000000     0 SECTION LOCAL  DEFAULT    4
     6: 00000000     0 SECTION LOCAL  DEFAULT    6
     7: 00000000     0 SECTION LOCAL  DEFAULT    7
     8: 00000000     0 SECTION LOCAL  DEFAULT    9
     9: 00000000     0 SECTION LOCAL  DEFAULT   10
    10: 00000000     0 SECTION LOCAL  DEFAULT   11
    11: 00000000     0 SECTION LOCAL  DEFAULT   12

显然是没编译pppapi.h中定义的api

查阅相关文件

C:\Users\rui.platformio\packages\framework-espidf\components\lwip\lwip\src\include\netif\ppp\pppapi.h

#include "netif/ppp/ppp_opts.h"
#if LWIP_PPP_API /* don't build if not configured for use in lwipopts.h */

...

/* API for application */
err_t pppapi_set_default(ppp_pcb *pcb);
#if ESP_PPP && PPP_AUTH_SUPPORT
void pppapi_set_auth(ppp_pcb *pcb, u8_t authtype, const char *user, const char *passwd);
#endif
#if PPP_NOTIFY_PHASE
err_t pppapi_set_notify_phase_callback(ppp_pcb *pcb, ppp_notify_phase_cb_fn notify_phase_cb);
#endif /* PPP_NOTIFY_PHASE */
#if PPPOS_SUPPORT
ppp_pcb *pppapi_pppos_create(struct netif *pppif, pppos_output_cb_fn output_cb, ppp_link_status_cb_fn link_status_cb, void *ctx_cb);
#endif /* PPPOS_SUPPORT */
#if PPPOE_SUPPORT
ppp_pcb *pppapi_pppoe_create(struct netif *pppif, struct netif *ethif, const char *service_name,
                                const char *concentrator_name, ppp_link_status_cb_fn link_status_cb,
                                void *ctx_cb);
#endif /* PPPOE_SUPPORT */
#if PPPOL2TP_SUPPORT
ppp_pcb *pppapi_pppol2tp_create(struct netif *pppif, struct netif *netif, ip_addr_t *ipaddr, u16_t port,
                            const u8_t *secret, u8_t secret_len,
                            ppp_link_status_cb_fn link_status_cb, void *ctx_cb);
#endif /* PPPOL2TP_SUPPORT */
err_t pppapi_connect(ppp_pcb *pcb, u16_t holdoff);
#if PPP_SERVER
err_t pppapi_listen(ppp_pcb *pcb);
#endif /* PPP_SERVER */
err_t pppapi_close(ppp_pcb *pcb, u8_t nocarrier);
err_t pppapi_free(ppp_pcb *pcb);
err_t pppapi_ioctl(ppp_pcb *pcb, u8_t cmd, void *arg);

#ifdef __cplusplus
}
#endif

#endif /* LWIP_PPP_API */

跳过本部分===

C:\Users\rui.platformio\packages\framework-espidf\components\lwip\lwip\src\include\netif\ppp\ppp_opts.h

#ifndef PPP_SUPPORT
#define PPP_SUPPORT                     0
#endif

...

/**
 * LWIP_PPP_API==1: Enable PPP API (in pppapi.c)
 */
#ifndef LWIP_PPP_API
#define LWIP_PPP_API                    (PPP_SUPPORT && (NO_SYS == 0))
#endif

C:\Users\rui.platformio\packages\framework-espidf\components\lwip\port\esp32\include\lwipopts.h

/*
   ---------------------------------
   ---------- PPP options ----------
   ---------------------------------
*/

/**
 * PPP_SUPPORT==1: Enable PPP.
 */
// #define PPP_SUPPORT                     CONFIG_LWIP_PPP_SUPPORT
#define PPP_SUPPORT                     1

MicroPython repo中sdkconfig.base中似乎可以设置是否启用PPP

# UDP
CONFIG_LWIP_PPP_SUPPORT=y
CONFIG_LWIP_PPP_PAP_SUPPORT=y
CONFIG_LWIP_PPP_CHAP_SUPPORT=y

pio中没找到相关文件,只有自动生成的sdkconfig.h,对应Makefile中

$(SDKCONFIG_H): $(SDKCONFIG_COMBINED)
	$(ECHO) "GEN $@"
	$(Q)$(MKDIR) -p $(dir $@)
	$(Q)$(PYTHON) $(ESPIDF)/tools/kconfig_new/confgen.py \
		--output header $@ \
		--config $< \
		--kconfig $(ESPIDF)/Kconfig \
		--env "IDF_TARGET=esp32" \
		--env "IDF_CMAKE=n" \
		--env "COMPONENT_KCONFIGS=$(ESPCOMP_KCONFIGS)" \
		--env "COMPONENT_KCONFIGS_PROJBUILD=$(ESPCOMP_KCONFIGS_PROJBUILD)" \
		--env "IDF_PATH=$(ESPIDF)"
	$(Q)touch $@

Makefile中去掉network_ppp.c,同时在modnetwork.c中去掉报错的那一行

STATIC const mp_rom_map_elem_t mp_module_network_globals_table[] = {
    { MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_network) },
    { MP_ROM_QSTR(MP_QSTR___init__), MP_ROM_PTR(&esp_initialize_obj) },
    { MP_ROM_QSTR(MP_QSTR_WLAN), MP_ROM_PTR(&get_wlan_obj) },
    #if !MICROPY_ESP_IDF_4
    { MP_ROM_QSTR(MP_QSTR_LAN), MP_ROM_PTR(&get_lan_obj) },
    #endif
    // { MP_ROM_QSTR(MP_QSTR_PPP), MP_ROM_PTR(&ppp_make_new_obj) },
    { MP_ROM_QSTR(MP_QSTR_phy_mode), MP_ROM_PTR(&esp_phy_mode_obj) },

至此pio中编译剩余两个FreeRTOS相关错误xQueueCreateMutexStatic

在C:\Users\rui\Documents\Data\Code\PlatformIO\MicroPythonEmbedded.pio\build\firebeetle32\config\sdkconfig.h添加

...
#define CONFIG_FREERTOS_TASK_FUNCTION_WRAPPER 1
#define CONFIG_FREERTOS_CHECK_MUTEX_GIVEN_BY_OWNER 1
#define CONFIG_FREERTOS_DEBUG_OCDAWARE 1
// added line
#define CONFIG_FREERTOS_SUPPORT_STATIC_ALLOCATION 1
// added line
#define CONFIG_HEAP_POISONING_DISABLED 1
#define CONFIG_HEAP_TRACING_OFF 1
#define CONFIG_LOG_DEFAULT_LEVEL_INFO 1
#define CONFIG_LOG_DEFAULT_LEVEL 3
...

====================================================================

发现上面这些东西可以直接通过改pio project目录下的sdkconfig搞定 添加下面几行

# PPPoS
CONFIG_LWIP_PPP_SUPPORT=y
CONFIG_LWIP_PPP_PAP_SUPPORT=y
CONFIG_LWIP_PPP_CHAP_SUPPORT
# FreeRTOS
CONFIG_FREERTOS_SUPPORT_STATIC_ALLOCATION=y

成功链接,报flash空间不足,暂时先不删除功能,直接改分区表

# Name,   Type, SubType, Offset,  Size, Flags
# Note: if you have increased the bootloader size, make sure to update the offsets to avoid overlap
nvs,      data, nvs,     ,        0x6000,
phy_init, data, phy,     ,        0x1000,
factory,  app,  factory, 0x10000, 0x180000,

烧录进入esp32打开串口调试后报错。

Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.
Core 0 register dump:
PC      : 0x400da13a  PS      : 0x00060930  A0      : 0x800d2744  A1      : 0x3ffbc360
A2      : 0x3ffbc380  A3      : 0x00000000  A4      : 0x3ffb46dc  A5      : 0xffffffff
A6      : 0x3ffaffac  A7      : 0x3ffbe45c  A8      : 0x800da13a  A9      : 0x3ffbc340
A10     : 0x00000000  A11     : 0x00000000  A12     : 0x0000041e  A13     : 0x3ffbf0c6
A14     : 0x00000000  A15     : 0x00000041  SAR     : 0x0000001f  EXCCAUSE: 0x0000001c
EXCVADDR: 0x00000010  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0x00000000

查看官方文档的解释

Guru Meditation 错误
本节将对打印在 Guru Meditation Error: Core panic'ed 后面括号中的致错原因进行逐一解释。
...
LoadProhibited, StoreProhibited
当应用程序尝试读取或写入无效的内存位置时,会发生此类 CPU 异常。此类无效内存地址可以在寄存器转储的 EXCVADDR 中找到。如果该地址为零,通常意味着应用程序正尝试解引用一个 NULL 指针。如果该地址接近于零,则通常意味着应用程序尝试访问某个结构体的成员,但是该结构体的指针为 NULL。如果该地址是其它非法值(不在 0x3fxxxxxx - 0x6xxxxxxx 的范围内),则可能意味着用于访问数据的指针未初始化或者已经损坏。

尝试debug报错Target disconnected.: Not a directory

把OpenOCD换成最新的release,debug恢复正常

debug发现每次都在.platformio/packages/framework-espidf/components/esp32/pm_esp32.c中卡住

    /* Configure all modes to use the default CPU frequency.
     * This will be modified later by a call to esp_pm_configure.
     */
    rtc_cpu_freq_config_t default_config;
    if (!rtc_clk_cpu_freq_mhz_to_config(CONFIG_ESP32_DEFAULT_CPU_FREQ_MHZ, &default_config)) {
        assert(false && "unsupported frequency");
    }
    for (size_t i = 0; i < PM_MODE_COUNT; ++i) {
        // =======================
        s_cpu_freq_by_mode[i] = default_config;
        // =======================
    }
}

每次执行mp_init()函数,点step into直接停在pm_esp32.cs_cpu_freq_by_mode[i] = default_config;这一行,因为没有.c文件。

貌似这段代码是在抛异常,应该和micropython无关。没法单步调试的话可能要根据call stack找问题出处。

在pio project的main.c中调用mp_initexecute_from_str都会报错,debug会定位到上面说的那行代码。execute_from_str代码如下

mp_obj_t execute_from_str(const char *str)
{
    nlr_buf_t nlr;
    // ===============================
    if (nlr_push(&nlr) == 0)
    // ===============================
    {
        qstr src_name = 1 /*MP_QSTR_*/;
        mp_lexer_t *lex = mp_lexer_new_from_str_len(src_name, str, strlen(str), false);
        mp_parse_tree_t pt = mp_parse(lex, MP_PARSE_FILE_INPUT);
        mp_obj_t module_fun = mp_compile(&pt, src_name, false);
        mp_call_function_0(module_fun);
        nlr_pop();
        return 0;
    }
    else
    {
        // uncaught exception
        return (mp_obj_t)nlr.ret_val;
    }
}

执行到nlr_push时会报错。nlr_push代码如下:

void nlr_pop(void) {
    nlr_buf_t **top = &MP_STATE_THREAD(nlr_top);
    *top = (*top)->prev;
}

很可能是MP_STATE_THREAD引起的错误,这是py/mpstate.h中定义的宏

extern mp_state_ctx_t mp_state_ctx;

#define MP_STATE_VM(x) (mp_state_ctx.vm.x)
#define MP_STATE_MEM(x) (mp_state_ctx.mem.x)

#if MICROPY_PY_THREAD
extern mp_state_thread_t *mp_thread_get_state(void);
#define MP_STATE_THREAD(x) (mp_thread_get_state()->x)
#else
#define MP_STATE_THREAD(x) (mp_state_ctx.thread.x)
#endif

mp_init()中调用了MP_STATE_VM,猜想是mp_state_ctx的问题

查看符号表

$ readelf -s libmicropython.a | grep "mp_state_ctx" -C 50

File: libmicropython.a(mpstate.o)

Symbol table '.symtab' contains 8 entries:
   Num:    Value  Size Type    Bind   Vis      Ndx Name
     0: 00000000     0 NOTYPE  LOCAL  DEFAULT  UND
     1: 00000000     0 FILE    LOCAL  DEFAULT  ABS mpstate.c
     2: 00000000     0 SECTION LOCAL  DEFAULT    1
     3: 00000000     0 SECTION LOCAL  DEFAULT    2
     4: 00000000     0 SECTION LOCAL  DEFAULT    3
     5: 00000000     0 SECTION LOCAL  DEFAULT    4
     6: 00000000     0 SECTION LOCAL  DEFAULT    5
     7: 00000004   816 OBJECT  GLOBAL DEFAULT  COM mp_state_ctx

查了一下Ndx列的含义

Ndx列表明每个符号所在的段,其中有三个伪段(pseudo section):ABS表示不该被重定位的符号,UNDEF表示在本模块引用,却在其他模块定义,COMMON表示还未分配位置的未初始化对象。

然后再次查询

$ readelf -s libmicropython.a | grep "mp_state_ctx"
     7: 00000004   816 OBJECT  GLOBAL DEFAULT  COM mp_state_ctx
    52: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    39: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    32: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
   342: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
   186: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
   118: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    24: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    96: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    34: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    50: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
   157: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    42: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    65: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    47: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    74: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    20: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    22: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    76: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    86: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    22: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    21: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    49: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    67: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    94: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
   116: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
   150: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    54: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    36: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx
    49: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND mp_state_ctx

除了mpstate.o中是COM其他出现mp_state_ctx的地方都是UND,很有可能是mp_state_ctx没初始化的问题。

如果执行的是mp_init错误是LoadProhibited,EXCVADDR: 0x00000008,如果直接执行execute_from_str错误是StoreProhibited,EXCVADDR: 0x00000010。

直接写micropython repo中main.c的代码会在mp_stack_set_top((void *)sp);抛异常,EXCVADDR: 0x00000000

mp_stack_set_top的具体实现:

void mp_stack_set_top(void *top) {
    MP_STATE_THREAD(stack_top) = top;
}

看一下mp_state_ctx的组成

typedef struct _mp_state_ctx_t {
    mp_state_thread_t thread;
    mp_state_vm_t vm;
    mp_state_mem_t mem;
} mp_state_ctx_t;

mp_state_thread_t

typedef struct _mp_state_thread_t {
    // Stack top at the start of program
    char *stack_top;

    #if MICROPY_STACK_CHECK
    size_t stack_limit;
    #endif

    #if MICROPY_ENABLE_PYSTACK
    uint8_t *pystack_start;
    uint8_t *pystack_end;
    uint8_t *pystack_cur;
    #endif

    ////////////////////////////////////////////////////////////
    // START ROOT POINTER SECTION
    // Everything that needs GC scanning must start here, and
    // is followed by state in the mp_state_vm_t structure.
    //

    mp_obj_dict_t *dict_locals;
    mp_obj_dict_t *dict_globals;

    nlr_buf_t *nlr_top;

    #if MICROPY_PY_SYS_SETTRACE
    mp_obj_t prof_trace_callback;
    bool prof_callback_is_executing;
    struct _mp_code_state_t *current_code_state;
    #endif
} mp_state_thread_t;

似乎都是对mp_state_ctx操作引起了问题,非常神秘。

然后我试着调用了一下结构体里面其他几个成员如stack_limit, dict_locals, dict_globals,

  void *p = MP_STATE_THREAD(dict_globals);
  printf("0x%d\n", (unsigned int)p);

EXCVADDR的地址貌似都对的上,都差0x4,基本确定是mp_state_ctx存在问题,但不一定只是mp_state_ctx存在问题。


本文章使用limfx的vscode插件快速发布