Strange networking issue (can ping, cannot telnet/ssh/ftp)

[damjanev]

Nothing fancy on my network. Huawei ONT/router from my provider acting as a WiFi access point, DHCP server and Internet gateway. No firewall or inspection functionalities on the LAN side.

I have several options to test during the weekend:

- play with the USB port on the wdtv for the network adapter
- use wired connection for both the laptop and the wdtv
- use direct connection with dhcp server on the laptop
- try the WiFi adapters

Update. Tried the other USB port, the wired connection to the router for the laptop and the direct wired connection. Same results.

Looking at the logs on the USB drive (log-saver.app.bin), i notice that the date and time changes during boot. Starts as 01.01.2000, and changes to the current date and time. I assume that an NTP client kicks in and sets the correct time and date. This means that some connectivity works. NTP and DHCP are UDP based protocols. Ftp, telnet and ssh are TCP. Another thing that i have seen in the logs is the checksum offload has been enabled. Can these two thing be connected?

[mad_ady]

Hmm, interesting... ICMP also works. If UDP really works you could start a reverse shell over netcat over UDP so you could connect instead of using telnet.

Regarding the checksum offload - yes, it could affect TCP, but it should affect all packets equally (including syn/ack). You could search for ways to disable checksum offload (though wireshark didn't report bad checksums).

The root cause might be connected to MTU - after the 3-way handshake larger packets are sent (though I need to check the capture). UDP DHCP/NTP traffic uses small packets. Although telnet uses small packets as well... If you get a reverse shell in we can try to poke around in /proc. If mtu is the issue ping with large packets should fail (you can try various sizes of payload)

[damjanev]

It is possible that TCP Offload "kicks-in" after the 3-way handshake. There are TOE modes that work like this. If ethertool is available, i will try to disable TOE in S00custom-options.
Is netcat available out of the box?

[damjanev]

Quick update. It responds to pings of up to 1472 bytes.

[damjanev]

S00custom-options seems to get loaded too early in the boot process. The network-log.txt from couple messages ago is quite empty

Code: Select all

lo        Link encap:Local Loopback 
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface


[mad_ady]

I'm not 100% sure for Gen1, but netcat should be there (at least as part of busybox):

Code: Select all

root@Deneb:/root# which nc
/usr/bin/nc
root@Deneb:/root# file /usr/bin/nc
/usr/bin/nc: symbolic link to `../../bin/busybox'
root@Deneb:/root# nc
BusyBox v1.19.2 (2011-09-23 16:53:42 PDT) multi-call binary.

Usage: nc [-iN] [-wN] [-l] [-p PORT] [-f FILE|IPADDR PORT] [-e PROG]

Open a pipe to IP:PORT or FILE

   -e PROG   Run PROG after connect
   -l   Listen mode, for inbound connects
      (use -l twice with -e for persistent server)
   -p PORT   Local port
   -w SEC   Timeout for connect
   -i SEC   Delay interval for lines sent
   -f FILE   Use file (ala /dev/ttyS0) instead of network



Unfortunately ethertool and mii-tool are not part of the firmware. It should be interesting to see if the asix driver module has some parameter that enables/disables TOE.

To get output out of S00custom-options, you can start a subshell and sleep for a while:

Code: Select all

(sleep 30; ifconfig -a > /tmp/ifconfig.log 2>&1)&


[damjanev]

I'm not 100% sure for Gen1, but netcat should be there (at least as part of busybox):

Unfortunately ethertool and mii-tool are not part of the firmware. It should be interesting to see if the asix driver module has some parameter that enables/disables TOE.

To get output out of S00custom-options, you can start a subshell and sleep for a while:

Code: Select all

(sleep 30; ifconfig -a > /tmp/ifconfig.log 2>&1)&


I seem to have missed this response and was about to ask if ethertool was available in the firmware and a way to run commands at a later boot stage. I will give this a try.
What is available in the firmware in /proc/net?

[damjanev]

I'm not 100% sure for Gen1, but netcat should be there (at least as part of busybox):

Unfortunately ethertool and mii-tool are not part of the firmware. It should be interesting to see if the asix driver module has some parameter that enables/disables TOE.

I briefly went through the official Asix Linux driver source code and there is no module parameter related to this.
There is one called bsize defined in asix.c.

Code: Select all

/* configuration of maximum bulk in size */
static int bsize = AX88772B_MAX_BULKIN_16K;
module_param(bsize, int, 0);
MODULE_PARM_DESC(bsize, "Maximum transfer size per bulk");

This one is interesting since it mentiones AX88772B the chip that my adapter uses). The constant is defined in asix.h as

Code: Select all

#define AX88772B_MAX_BULKIN_16K      4

with a struct below:

Code: Select all

struct {unsigned short size, byte_cnt, threshold; } AX88772B_BULKIN_SIZE[] = {
   /* 2k */
   {2048, 0x8000, 0x8001},
   /* 4k */
   {4096, 0x8100, 0x8147},
   /* 6k */
   {6144, 0x8200, 0x81EB},
   /* 8k */
   {8192, 0x8300, 0x83D7},
   /* 16 */
   {16384, 0x8400, 0x851E},
   /* 20k */
   {20480, 0x8500, 0x8666},
   /* 24k */
   {24576, 0x8600, 0x87AE},
   /* 32k */
   {32768, 0x8700, 0x8A3D},
};


There is another one called msg_level defined in axusbnet.c

Code: Select all

/* use ethtool to change the level for any given device */
static int msg_level = -1;
module_param(msg_level, int, 0);
MODULE_PARM_DESC(msg_level, "Override default message level");

and that is it. Maybe the driver version in the app.bin is buggy

Netcat shell, bound to an UDP port seems like the best troubleshooting option at the moment.

[mad_ady]

Go for it! I'm grabbing my popcorn

If we can't figure out what the cause of the problem is, you could set up tunnels over UDP and still get to use HTTP/FTP/Telnet/SSH. See if any of these work (once you get access):

http://serverfault.com/questions/7735/tunnel-over-udp-or-tcp
man7.org/linux/man-pages/man8/ip-l2tp.8.html